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The purpose of the study was to investigate the effects of training and 
proficiency in public speaking on the dimensionality of speech evaluation. Dimensionality 
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I 



INTRODUCTION 



1.1 Purpose of the Study 

The purpose of this study is tg investigate the effect of training 
and proficiency in public speaking on the dimensionality of speech 
evaluation. Specifically, and in terms of researchable objectives;, 
this study is aimed at the following; 

1. The determination of specific dimensions of 
speech evaluation as measured by students 
of public speaking in their attempts to make 
objective distinctions among varying qualities 
of public address. 

2 . The determination of the nature of the relation- 
ship between training in public speaking and 
the recognition and use of dimensions of speech 
evaluation which make objective distinctions among 
varying qualities of public address . 

3. The determination of the nature of the relationship 
between developed proficiency in public speaking 
and the ability to recognize and use objective 
dimensions of speech evaluation. 



1 . 2 Rationale for the Study 

The problems associated with speech evaluation have been among 
the most perplexing and persistent ones in speech education. This 
area of concern clearly relates to the classroom where attempts to 
improve the student speaker often take the form of a critique of his 
speaking by in instructor and peer evaluators. It is the assumption 
of such a practice that the critique will provide a clear explanation 
of what the student did well and/or badly in the particular speaking 
situation, and that given this explanation, the student, by replication 
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of the good and avoidance of the bad traits , will be able to become a 
more effective speaker. This approach to the training of public 
speakers obviously depends upon the student speaker's ability to 
identify in his own mind those criteria which clearly differentiate 
between qualities of speaking and then to use these scales as 
guidelines for the development of his own speaking skills. 

Furthermore, the student should be able to reinforce the use of 
these evaluative criteria by applying them to speeches of varying 
quality other than his own. 

This concern for the problems associated with speech evaluation 
also relates to the basic rationale of the discipline of Speech itself. 

It seems implied by the various course listings beyond basic speech 
which emphasize the theory of and practice in public address, that a 
detailed study of specific types of speaking will influence a student's 
perception of the communication act and his ability to effect change. 

The dimensionality of the communication act clearly involves an ability 
to make judgments and any influence on that dimensionality must 
necessarily affect how a student will evaluate public speaking. 

A concern for the dimensionality of speech evaluation also 
extends beyond the classroom to a world that gives the individual 
in a communicating society the primary role of evaluator of those 
who seek to modify his behavior for some end. Research has further 
defined this role of evaluator by pointing out that the major involvement 



er|c 



1-3 



of an individual in the communication act is as a listener to oral 
expression. 

That the concern for speech evaluation has not motivated 
significant research on the associated problems is a critical comment 
on the field. To a very real extent , Speech is a late comer in the 
application of the tools of social science. Yet it is apparent that such 
tools are necessary to the determination of any -igorous solutions to the 
perplexing problem of speech evaluation. However, even with the tools 
in hand, the speech researcher is immediately faced with the dilemma of 
the vast amount of variability that exists in the ways in which Speech is 
taught. The most dedicated and competent social scientist may investigate 
speech evaluation at his own institution and have his findings rejected 
on the grounds that the approach to speech evaluation used is unique to 
that institution or to a particular instructor within that institution. 

That such variability exists points to the persistence of the problems of 
speech evaluation. 

1.3 The Method of Attack 

The study reported on the following pages represents an attempt 
to use the modem tools of social science in a massive attack on the 
problems of speech evaluation. In all approximately 4800 students 
representing eight different universities participated in the study. 

These students yielded more than 14,400 evaluations of speeches 
representing three distinct quality levels. The gathering of data for 






1-4 




the project took about one year and was accomplished only through the 
cooperative efforts of the institutions involved. Those were Michigan 
State University, Iowa State University, Ohio State University, and the 
Universities of Wisconsin, Connecticut; Minnesota, Illinois, and 
Michigan, Appendix A contains a list of the descriptions of the 
courses from which the project data were evaluated. 

The study was designed to use the statistical technique of factor 
analysis as an aid for the interpretation of the data , This technique 
examines the intercorrelation of evaluative criteria with a view to 
finding the significant dimensions of judgment. For the purposes of 
the study a dimension shall be defined as a cluster of evaluative 
criteria highly correlated with one another and less correlated with 
clusters of criteria representing other dimensions. There will be 
further discussion of the technique of factor analysis in latter sections 
of this report. It is sufficient to say at this point that factor analysis 
appears to be an invaluable tool in attacking the problems associated 
with speech evaluation. 

Another tool used in this project was the Control Data Corporation 
3600 model computer. Certainly it is this tool that made it possible to 
apply the techniques of factor analysis to the many combinations of data 
associated with the objectives of the research. The computer has provided 
ready access to particular sets of data and facilitated the checking, 
scoring, and storage of the 14,400 speech evaluations. 
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Finally and most importantly, the research to be described in the 
following pages had its financial support in the form of a grant from the 
United States Office of Education. While some will argue the priority 
of such support when there is a multitude of projects clamoring for 
funds, this writer feels that acceptance by reviewers, consultants 
and officers of the United States Office of Education of a project 
aimed at a significant problem within the discipline of Speech is 
an invigorating event for our field, 

1.4 Design of Report 

The remainder of this final report will be divided into four 
sections: The Preparation of Research Materials for the Project; 

The Logistics and Statistical Design of the Project; The Results of 
the Project; and Conclusions and Recommendations for Further Research. 
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THE PREPARATION OF RESEARCH MATERIALS FOR THE PROTECT 
2.1 Film Preparation 

In order to provide the students involved in the project with 
experience in speech evaluation it was decided to film some student 
speeches of varying quality. These films would then be shown to 
students from whom data would be gathered in order to secure their 
basis of classification through the evaluative criteria they deemed 
relevant to public speaking. 

Students enrolled in Speech 305, Section 4 (Persuasion) for the 

A 

Fall term of 1966 at Michigan State University were used in the 
preparation of the filmed public speeches to be used in the project. 

The students involved in this phase of the project were informed 
exactly what was to happen and how the films were to be used. They 
were asked to sign a release (see Appendix G) authorizing the recording 
and viewing of the films for the purposes of the research project. Wo 
students refused to participate in this phase of the project. Had any 
done so, they would have beel transferred to another section of the same 
course with a different instructor. Mention of this point is made to 
establish in the mind of the reader that: (1) the students whose filmed 
speeches were to be used in the project were fully aware of the intended 
use; (2) the students were given an opportunity not to participate 
with no penalty; (3) the students signed releases of authorization 
of the taking of and subsequent use of films of their speeches. 
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The filming of the speeches took place in an especially designed 
classroom in the Michigan State University Audio-Visual Center. All 
sessions of the class met in these surroundings. The students first 
gave a speech that was not filmed. This was to acquaint them with the 
parameters of the physical setting under which the films were to be 
taken. As these first non-filmed speeches were given, the instructor 
attempted to identify particular types of speaking behavior which he 
desired to have the student duplicate on the occasion of the actual filming. 
A specific attempt was made to capture those aspects of public 
speaking which appeared to be related to the evaluative criteria contained 
on the speech rating form developed for the project. 

In all, 19 films were made, one for each student in the class. 

The speeches filmed were all intended to be persuasive in nature. They 
were five to eight minutes in length and on topics selected by the 
students. All films were made in two class sessions. Dual track sound 
recordings were made and transferred to the films at the time of processing. 
This process assured a high quality of sound reproduction for the films 
used in the project. 

2.2 Film Selection 

The purposes of the project required the use of twelve filmed 
public speeches. The films were to be divided into four sets of three, 
each set representing three distinct quality variations . Once the films 
of the 19 student speeches had been developed, the process of film 



selection was begun. In order to make the selections, members of the 
Rhetoric and Public Address staff and selected graduate students of 
Michigan State University* were asked to view the films and evaluate 
them in terms of the criteria contained on the speech rating form. The 

4 

19 films were evaluated by six raters in three sessions of approximately 
one hour each. The speeches were shown to the evaluators in the same 
order in which they were recorded. Because of the time length of the 
speeches the number viewed at each rating session was varied. In 
the first rating session for the purposes of selection six films were 
viewed, in the second session five films were viewed, and in the third 
and final session eight films were viewed and evaluated. 

In each rating session during the process of film selection the 
evaluators were asked to judge each speech according to the criteria 
contained on the speech rating form to be used in the project. These 
ratings were to be based on a one to seven scale with 1 representing 
a low and 7 a high rating. In addition the evaluators were asked to 
classify the films which they saw as being low, middle, or high 
quality as compared to the type of speaking they were used to hearing 
in their own classrooms. 

*Twelve members of the Rhetoric and Public Address staff who participated 
in the selection of the filmed speeches to be used in the project had 
previous experience in classroom speech instruction. In fact, most 
had been instructors in the basic speech course at Michigan State 
University. No one directly connected with the project was involved 
in the film selection aside from arranging the logistics of the viewings. 
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Using the statistical procedure called the Intraclass Corr elation. 

(see reference 5) the reliability of the six raters for each session was 
determined for each of the evaluative criteria and the classification 
variable (see reference 6). Table 2.2-1 represents the combined rater 
reliability on each evaluative criteria for the three rating sessions used 
in the process of film selection. 

Using essentially the same procedure for determining the 
degree of classification consistency reliability coefficients of .95/ 

.93/ and .87 were obtained for rating sessions 1/ 2 , and 3 respectively. 

\ ^_^ince the judging reliability on the classification scale was quite high, 
that variable was used to select the twelve films on which further 
analysis would be run. The films so determined were those on which a 
consensus of judgment with respect to the classification variable was 
reached, and which, in keeping with the needs of the project, were of 
varying quality. The relatively high reliability of the judges across 
rating sessions on the evaluative criteria contained in Table 2.2-1 
would seem to yield some justification for selecting films from each 
session for further analysis. 

Table 2.2-2 represents the twelve films selected for further analysis 
according to the session in which they were evaluated as well as the 
quality (low, middle, high) classification they represented as judged 
by a consensus of the evaluators . 
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The further analysis of the films was in the form of comparing 
the within classification group rating variance tu the between classification 
group rating variance on each of the evaluative criteria (see reference 10) . 
This procedure yielded statistically significant (.05) differences among 
the film classifications on each of the 19 evaluative criteria. It should 
be noted that an assumption was made that the judge reliability on the 
evaluative criteria per rating session was sufficient to consider the 
raters involved in the process of the film selection to be interchangeable. 
Table 2.2-3 represents the means, and standard deviations for quality 
groupings of the films represented in Table 2.2-2. 

As a result of the additional analysis it was decided that the 
12 films representing the consensus classification judgments of the 
rathers would be those used in the main research project. However, 
one problem remained for consideration, namely, the grouping of the 
selected films. The requirements of the project (as set forth in the 
project proposal) were such as to necessitate the grouping of the 
selected films into four sets (designated A, B, C, and D) of three 
films each (one film representing each of the classifications Low, 

Middle and High) . In order to apply some degree of rigor to this 
process of film grouping the ratings given to the selected speech 
films were further analyzed using the statistical technique of 
Discriminant Analysis (see reference 10) . Here the attempt was made 
to determine the number of discriminant functions which would maximize 




the ratio of among to within classification group dispersion. Thus they 
would allow the plotting of the status of each film with respect to each 
discriminant function. This process then allowed the films to be 
grouped according to the requirements of the project in such a manner 
that each grouping tended to approximate the classification group 
dispersion. 

Table 2.2-4 represents the assignments of the films to their 
respective sets by film number. Appendix B contains the student 
outlines of the supposed content of the speeches. 

Once the films were assigned to their sets prints were obtained 
in sufficient number to allow each set of films to be shown in three 
different orders (each quality classification appearing once in first, 
second, and third positions). Further discussion of the various manipulations 
of the films will be handled in the section of this report devoted to the 
logistics of the project. 

2.3 Development of the Questionnaire 

In conjunction with that portion of the study dealing with the 
effects o;. training on the dimensionality of speech evaluation a student 
questionnaire was developed for the purposes of ascertaining: (1) The 
amount of academic training in speaking the student had prior to his 
participation in the project; (2) The student's self-perception of his 
abilities as a public speaker with particular reference to the evaluative 
criteria contained on the rating forms to be used by the student in the 
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evaluation of the filmed student speeches. In addition the questionnaire 
contained items which would aid the classification and storage of the 
study data. 

The questionnaire (which later became the front side of the rating 
form) was pretested in the Fall of 1966 at Michigan State University. 

This pilot study took the form of the administration of the questionnaire 
to 326 students in the beginning speech courses and 83 students in 
advanced speech courses at Michigan State University. The objective 
of the pilot study was to ascertain the degree of difficulty in understanding 
what v/as being asked as reflected in mismarking the questionnaire. The 
results of the questionnaire were compared to other existing University 
records yielding similar data. For the most part the pilot study analysis 
with respect to information relating to student classification and previous 
training corresponded, closely to the. data yielded by other instruments 
not related to the project. One area which appeared to give some difficulty 
was that of correctly designating the major. It was decided# as a 
result of the pilot study# that this problem was best handled at the time 
of administration. It will be noted in a later section of the report 
dealing with the logistics of the project that special instructions 
to the administrator of the questionnaire were given. Appendix J 
contains a copy of the items comprising the questionnaire. 

2.4 Development of the Self-Perception Scales 

As noted previously# the students who participated in the pilot study 
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were asked to give their perception of themselves as speakers using 

the same criteria that would be used to evaluate the filmed speeches. 

The main purpose of this self-perception evaluation was to familiarize 

the students with the criteria as well as the format of the scales . It 

was felt that a self evaluation would provide the student with a meaningful 

example from which to make a judgment of other speakers. It was also 

reasoned that the data from the self evaluations would provide some clue 

as to the dimensionality of the rating instrument. Data from this particular 

phase of the pilot study were subjected to two types of analysis . The 

first analysis was simply a tabulation of the student responses to the 

individual scale items . This procedure allowed a determination of 

mismarking (a mis-mark being any character not between 1 and 7 

inclusively). As a result of this analysis it was determined that mismarking 

was not an appreciable problem with respect to student responses to the 

evaluative criteria contained on the rating form and in the format represented 

by it. The second statistical procedure employed on the self-perception 

^ • 

rating scales was that of factor analysis. Table 2.4-1 represents the 
means and standard deviations of both the beginning and advanced 
students for the 19 self-perception criteria used in the pilot study. 

Table 2.4-2 represents a four factor solution for the pilot study self 
perceptions of beginning speech students at Michigan State University . 

Table 2.4-3 represents a five factor solution for the pilot study 
self-perceptions of students in advanced courses in public speaking 
at Michigan State University. 
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The results cited in Tables 2.4*"2 and 2.4-3 indicated two things to 
the researchers: (1) students making judgments of themselves as public 
speakers using the evaluative criteria listed conceived of the rating 
instrument as multidimensional; (2) that the diinensionality of the 



rating instrument when used to reflect self perceptions could indicate 
a difference between students according to their training . Thus by 
evaluating his self perception as a public speaker the student could 



familiarize himself with the use of a multidimensional rating instrument. 



The above discussion represents the rationale for the student 
questionnaire and its content format. Appendix J contains the self- 
perception scales. 

The questionnaire and the self perception rating scales will be 
hereafter referred to as the front side of the project rating form (Appendix J) . 



2.5 Development of the Speech Rating Instrument 

In September of 1966 word was begun on the expansion of the 
three dimensional rating scale used in the beginning course in public 
speaking at Michigan State University to incluc|e additional factors 
deemed relevant to the act of speech evaluation. Appendix D contains 
the criteria and format for the rating scale then in use at Michigan 
State. For the purpose of providing perspective on the rationale 
surrounding the use of rating scales in the evaluation of public speaking 
the following review of research in the area is offered. 



o 
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2 . 6 Previous Research Dealing with Speech Rating Instruments 

The field of Speech has' long been concerned with the problems 
surrounding the evaluation of public speaking. This concern is 
evidenced by the many attempts to develop adequate speech rating 
scales for use both in and outside the classroom (Stevens, 1928; 
Norvelle, 1934; Monroe, Remmers, and Lyle, 1936; Bryan and Wilkie, 
1941; Thompson, 1943-44; Fotheringham, 1956). However, it is 
relatively recently that researchers have approached the problem of 
speech evaluation from the point of view of factor analytical dimensions 
of the act. In 1957, Brooks used factor analysis as a technique for the 
identification and classification of items to be used on a forced- 
choice speech rating scale. While he made no attempt to establish the 
concept of dimensionality to the act of speech evaluation. Brooks' 
work did point to a methodology that appeared highly relevant to the 
issues surrounding that act. In 1962, Becker used the techniques of 
factor analysis in an investigation into rating scale independence. 

He was able to clearly establish that certain items traditionally 
associated with speech evaluation tended toward homogeneity , 
but could be clustered in such a manner as to produce heterogeneous 
factors of speech evaluation. He labelled '.these factors: "Content- 
analysis", "delivery", and "language". 

Two studies in 1965 involved the use of factor analysis in order 
to select items for a rating scale. Price was interested in developing 
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an instrument which could be used to measure the speech attainment 
of students desiring to avoid taking the required course in basic public 
speaking at the University of Wisconsin. He went through the recent 
literature in the field of speech relating to the principles and concepts 
of public speaking and came up with thirty-four scale items. Using 
instructors of public speaking as evaluators and the thirty-four scale 
items as a basis for rating student speakers. Price was able to identify, 
vie the techniques of factor analysis, six factors of speech evaluation. 

They were: reasonableness, intelligibility, bodily action, social acceptability, 
language, and voice. He was also able to demonstrate that a high degree 
of reliability could be obtained on these factors by pooling the ratings 
of the speech evaluators. Johnson, Lashbrook, and Ralph were 
interested in developing a speech rating scale based on student 
generated items . They used factor analysis to first discover the 
dimensions of student evaluation and then used these results as a ^ 

basis for the selection of items to be included on a speech rating scale. 

Results of this investigation indicated that students in the first half of a 
basic course in public speaking tend to evaluate speeches on three 
factor dimensions. These were labelled: “materials of development", 

"materials of experience", and "personal proof. It is this study that 
produced the rating scale in use in the basic course at Michigan State 
University (see Appendix D). 

The results of this study confirmed the findings of Price with 
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respect to the reliability of factor analytically selected scale 
items. In an effort to determine the objectivity of their scale, Johnson, 
Lashbrook, and Ralph had students evaluate filmed speeches of 
known varying qualities to see if the items could be used to make 
distinctions between the presentations. Assuming no more than 
ordinal scaling, they showed that students were able to rank the 
speeches as "high", "middle", and "low", and that this ranking 
corresponded to that coming from the independent determination of 
the quality of the filmed presentations at another university. 

In general, it could be said of the research in the field of 

speech with respect to speech evaluation: 

1. Dimensions of speech evaluation do exist and are 
identifiable by factor analytical techniques . 

2 . The instruments for measuring these dimensions have 
* not been yet fully developed. Items on the rating 

scales and factor labels tend to vary with the researcher 
and the objective of his study. 

3 . The variance in the number of dimensions between 
studies might be accounted for in terms of speech 
training and/or the material from which scale items 
are selected, 

4. Methods do exist for determining the objectivity 

of dimensional speech evaluation, but little has been 
done to determine the influences of multidimensionality 
on scale objectivity. 



2.7 Selection of Items to be Used on Speech Rating Form 

As was mentioned previously, an attempt was made in the Fall 
term of 1966 at Michigan State University to expand the rating scale 
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developed by Johnson, Lashbrook and Ralphs The research previously 
done on the scale had indicated that it could represent a three 
dimensional speech evaluation instrument and account for as much as 
75% of the variance when used by students to evaluate filmed as well as 
live classroom speech performance* It appeared, furthermore, that 
the three dimensional aspects of the scale seemed to hold given 
varying qualities of the performance being judged. Using the results 
of Price , who arrived at six factor analytic dimensions of speech 
evaluation, it was decided that the best method for increasing the 
dimensionality of the then Speech 101 rating instrument was to incorporate 
some of the items used by Price to represent specific factors not 
represented on the existing scale. Price had found a detectable area of 
speech evaluation which he labelled as the language factor. Using the 
standard rhetorical concept of style as the best language correlate the 
items word choice , vividness , imagination a nd interest were added 
to the scale. Price had also found a dimension of speech evaluation which 
he labelled the voice factor. In order to incorporate this dimension into 
the then existing scale the following items were added: diction ,. 
vocal inflection, and fluency . Finally, an attempt was made 
to add a visual aspect to the rating scale. In the development of 
the original scale used in the beginning speech course at Michigan 
State University, Johnson, Lashbrook and Ralph had attempted to 
isolate the visual aspect of speech evaluation but with little success, 
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However, it was felt that another attempt should be made to 
represent this factor. Using the factor analytic results of ooth the 
Price and the Johnson, Lashbrook, and Ralph studies, the following 
items were selected to represent the visual aspect of speech 
evaluation: physical appearance and bodily movement. 

It was hoped that the addition of the above stated items would 
yield somewhere between a three and six dimensional rating form. 

Such a range was viewed as sufficient for the project particularly if 
the number of dimensions of the rating form used in the actual 
evaluation of speeches appeared to vary within that range. 

2.8 Factor Analysis Results of the Use of the Rating Form in the 

Pilot Study 

Once the decision was made as to what items should be included 
on the speech rating form, an attempt was made to explore the dimension- 
ality of the instrument by having students in both beginning and 
advanced courses in public speaking at Michigan State University 
use the scale in the. evaluation of classroom performances. This phase 
of the pilot study came approximately five weeks into the term for the 
respective courses. In total 453 evaluations were made by students 
in the beginning course and 192 evaluations by students enrolled in 
advanced courses.* 

*The beginning course in public speaking at Michigan State University 
is Speech 101-Public Speaking. The advanced courses used to provide 
pilot study data were Speech 305-Persuasion and Speech 309-Argumen- 
tation . 



Table 2.8"1 represents the means and standard deviations for 
the respective scale items as used in the pilot study. It should be 
noted that the students in the advanced courses appeared to give 
consistently higher ratings to the speeches they observed than did 
the beginning speech students. However, there was no consistent 
difference between the standard deviations of the two training levels 
represented. This result would seem to be expected since students 
enrolled in the advanced courses had almost all completed a basic 
course in public speaking. It is probably also the case that the 
students who enroll in advanced courses probably did well in the begin- 
ning prerequisites to that course. It is interesting to note that 
advanced students also rated themselves consistently higher than 
those students in the beginning course (see Table 2.4-1). 

Table 2.8-2 represents a three factor solution for the beginning 
speech students and Table 2.8-3 a three factor solution for the 
advanced students who participated in this phase of the pilot study . 

A three factor solution for both training levels was first examined 
in order to determine what effect the additional items would have on 
the factor structure of the items appearing on the original Speech 101 
scale (see Appendix D) . It was concluded that the effect of adding 
items to the original rating form did allow for a slight redefinition of 
factor content, but the result was not in terms of reducing dimensionality. 
It was decided to continue to extract factors by rotation for the data 
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of the beginning and advanced students as long as a meaningful 

A 

factor content could be established. 

Table 2.8-4 represents the four factor solution for the same 
data represented 'by Table 2.8-2. The four factor solution seemed 
most appropriate for three reasons: First, it was the solution that 
yielded the largest number of factors represented by at least two 
items loading highest on a dimension. Second, the self-perceptions 
of the beginning speech students also suggested a four factor solution. 
Third, the extraction of a fourth dimension resulted in the least 
reduction of factor content when compared to the three factor solution 
for the same data. 

Tables 2.8-5 and 2.8-6 represent four and five factor solutions 
respectively for the data stemming from the advanced students who 
participated in the pilot study. Both of these solutions were examined 
in order to establish a base of comparison with those results stemming 
from data provided by the beginning students who participated in the 
pilot study. However, using the same criteria cited in the establish- 
ment of a meaningful factor solution for the beginning students it is 
spparent that the advanced data are best represented by Table 2.8-6. 

It should also be noted that a five factor solution for the advanced 
data accounted for approximately the same amount of variance as the 
four factor solution for the beginning students . 




The purpose of the factor analyses of the pilot study was two-fold: 
(1) to determine whether the items selected to be included on the rating. 
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form would represent somewhere between three and six dimensions 
of speech evaluation; (2) to determine whether the content of the 
factors within given solutions as well as the strength of those 
factors suggested differences between rating groups . This two-fold 
purpose appears to have been met. 

2 . 9 Section Summary 

In order to determine the effects of training and proficiency on 
the dimensionality of speech evaluation, several different types of 
research instruments needed to be developed. The researcher was 
most interested in determining the influence of the major variables 
as they affected student judgment of varying qualities of public 
address . 

Considerable time and expense went into the development of 
films of student speeches. The objective was to select 12 films 
representing three qualities of student speaking: high, middle, and 

low. The process of film selection involved teachers of public 

> 

speaking who viewed the films and made judgments by criteria 
eventually to be included on the speech rating form of the project. 
The films finally selected for the project represented the most 
consistent judgments of the reviewers with respect to the nominal 
classifications of high, middle, and low qualities of presentation 
as well as ratings given on the evaluative criteria selected for 
inclusion on a speech rating form. 
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In order to determine the previous training of the students who 
would eventually provide data for the project a questionnaire was 
developed. This questionnaire was administered in a pilot study 
to determine whether there were any avoidable problems with respect 
to marking the instrument. Some minor problems were discovered 
and the proper format adjustments made. As part of the questionnaire 
a student's perception of himself in terms of the criteria to be used 
on the speech rating was sought. While this self perception was 
not integral to the purposes of the study it was considered by the 
researcher to be a clue to the dimensionality of the evaluative criteria 
as well as a method for familiarizing the students with the rating 
instrument to be used in the film evaluations. 'The self-perception 
data from the pilot study were subjected to factor analyses . 

The development of the actual speech rating form was based for 
the most part on previous factor analytic studies done at the University 
of Wisconsin and Michigan State University. The items selected for 
inclusion on the rating instrument were aimed at the establishment of a 
multifactoral approach to speech evaluation. The items were placed 
on a rating form and administered as part of the pilot study to students 
in beginning and advanced courses in public speaking at Michigan 
State University. The students were asked in the case of the pilot 
study to evaluate the actual classroom performance of their peers. 
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As a result of the factor analyses of the pilot study data the ■ 
following conclusions were drawn with respect to the developed 
rating scale for the evaluation of public speaking. 

(1) The addition of items to the three dimensional 
scale developed at Michigan State University 
prior to the pilot phase of the project did not 
reduce the dimensionality of the instrument. 

(2) There was sufficient evidence to indicate that 
the developed instrument yielded as many as 
five factor solutions which, given the appropriate 
criteria, (to be discussed in a later section of 
this report) might well point to differences between 
rating groups. 

(3) The contents of the factors within particular 
solutions were sufficiently variable to suggest 
differences between rating groups . 

(4) The strength of particular factors in terms of 
accountable variance was sufficiently variable 
to suggest differences between rating groups. 

The above conclusions seemed in keeping with the intended 
objectives of the proposed research. Therefore, it was decided to 
use the questionnaire, self perception scales, and rating form as 
developed in the pilot study. It should be noted that as these 
instruments were being developed so were the procedures for scoring 



and handling of data. 
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TABLE 2.2-1 

Judging Reliability Per Rating Session 
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TABLE 2.2-2 

Selected Filins by Quality Classification and Bating Session 
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the process of film selection 
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The Logistics and the Statistical Design of the Project 

3.1 Procedures for Involving Universities in the Project 

Once the researcher was relatively assured that films, 
questionnaire, and rating form were in their final stages of 
development, consideration was given to the procedures under which 
institutions other than Michigan State University could be involved 
in the project. At a meeting of the Research Board of the Speech 
Association of America held in East Lansing in September 1966, 
the members of that board agreed to aid the researcher in enlisting 
universities and colleges in the project. The original intent was to . 
involve the Big 10 universities only. However, one member of the 
research board suggested that such a conference parameter was not 
necessary to the project and that since project participation involved 
scheduling around normal academic activities the researcher would 
be wise to use available and established Departments of Speech 
regardless of their various alignments . The researcher decided that 
he was under some obligation to stay within the confines of the original 
proposal and would extend beyond the Big Ten if those institutions 
in that conference were unable or unwilling to participate in the project. 

In November of 1966 a letter was addressed to the chairman of 
each Department of Speech in the Big Ten Conference. This letter 
briefly explained the project and its objectives and asked the 
addressees whether their departments would be willing to participate. 
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The letter went on to ask each chairman to designate a member of 
his staff who could be contacted by the researcher with respect 
to,the possible involvement of his department in the project. 

Appendix E represents the content of the initial letter of request.* 

As a result of the initial letter of request four institutions indicated 
that they would be unable to participate in the project during the 
1966/67 academic year. The remaining institutions expressed a 
willingness to participate if their schedules and that of the project 
would permit. Each institution willing to participate designated a 
member of its staff to discuss the logistics of the project with the 
researchers at the 1966 Convention of the Speech Association of 
America to be held in Chicago . 

After meeting with the representatives those Speech Depart" 
ments whose chairmen had agreed to participate in the project, it 
was determined that only in the cases of seven of the institutions 
could the schedule of the project be brought in line with the academic 
calendar of the departments. It was decided after the SAA Convention 
to attempt to recruit other institutions than those in the Big Ten in 
order to fulfill the purpose of the project with respect to adequate 
sample size and academic variability . Three criteria were used to 
select those institutions which would be asked to participate in the 
project beyond those covered by the original letters of request. 

*The initial letter of request was also sent to the University of Pittsburgh 
because of the desire of its then Chairman, Dr. Jack Mathews, to 
participate in the project. 
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These criteria were: 

(1) The academic calendar of the institution in 
relation to the intended logistics of the project. 

(2) TLe availability of students in both beginning 
and advanced courses in public speaking that 
involved performance,* 

(3) The willingness .of a member of the staff of the 
institution to supervise the administration of 
the project. 

As a result of the application of the above criteria two additional 
institutions were added to the seven previously enlisted. Appendix- E 
contains a list of representatives of those institutions ready to 
participate in the project. 

3.2 The Logistics of the Project 

The original intent of the project was to have students 
enrolled in the basic public speaking course evaluate the prepared 
films using the developed rating scale. They were to evaluate one 
set of films at the beginning of the course and another set at the 
end. In addition, students enrolled in advanced courses in public 
speaking (courses with a prerequisite of a basic course in public 
speaking) at the participating institutions would be asked to evaluate 
a set of the filmed speeches at the end of their course. The design 
of the research was such that all 12 films would be subjected to 

^This criteria was later to rule out the inclusion of one institution 
which originally agreed to participate in the project. 
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evaluation at all the training levels represented by the students 
involved in the research. 

Logistically, the gathering of data was divided into two 
phases. The first phase of the project (designated the pretest) 
involved students enrolled in the basic speech courses at participating 
institutions . The pretest was administered during the first week 
of classes. Approximately half of the students constituting this 
population were shown a set of three films , each set representing 
high/ middle , and low qualities of speaking. For the pretest each 
school was sent three reels of film. Each reel contained the films 
of a given set arranged in a different order for showing. Appendix F 
represents the instructions that were used in the administration of 
the pretest at participating institutions. Table 3.2-1 represents 
the assignment of film sets to the various schools in the pretest 
phase of the project. 

The second phase of the project (designated the posttests) 
involved: (1) students enrolled in a basic public speaking course 
who had participated in the pretest research; (2) the students enrolled 
in the same courses as represented- in the phase I but who had not 
participated in the pretest; and (3) students enrolled in advanced 
public speaking courses at the participating institutions. The 
posttests were administered during the final week of classes. 
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Table 3.2-2 represents the assignment of film sets to the 

■4 

basic stucents who did not participate in the pretest. This division 
of the posttests was designated posttestl. It will be noted that the 
films shonm in posttestl were the same as those shown in the pretest 
for the respective institutions. 

Table 3.2-3 represents .the assignment of film sets to the 
basic students who had participated in the pretest.' This division 
of the posttests was designated posttest2 . It will be noted that the 
films shown in posttest2 were different from those shown in the pretest 
for the respective institutions . 

i 

Table 3.2-4 represents the film set showings for phase II 
of the project to those students enrolled in advanced courses in public 
speaking at the institutions participating in the project. This division 
of the posttests was designated the advanced condition of the research. 
It will be noted that the films shown in the advanced condition were 
the same as those shown in the pretest and posttestl conditions for 
the respective institutions. 

Appendix G represents the instructions that were used in the 
administration of phase II of the project at participating institutions. 

The logistics of the project were such ?s to represent three 
training levels in public speaking: (1) students prior to training 
in a basic course in public speaking; (2) students immediately after 
training in a basic course in public speaking; and (3) students 
immediately after ^ aining in an advanced course in public speaking 
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for which a basic course was prerequisite. Furthermore, for one 
of the film sets shown at each institution the speeches were 
evaluated by students at all training levels.* For each institution 
there was a replication of the posttest involving a second set of 
films for students completing the basic course. This would allow 
for an examination of the effects of learning to use the rating form as 
opposed to actual training received in public speaking. Finally, 
taking the institutions as a group, all film sets were used in all 
four data-gathering conditions . 

i 

The questionnaire Snd self-perception scales were filled out 
by the students for the pretest, posttestl, and advanced conditions 
of the research . 

One of the purposes of the project dealt with determining 
the effect of proficiency in public speaking on the dimensionality 
of speech evaluation. The attainment of this purpose involved the ' 
use of the final grades that the students received in their respective 
courses as the measure of speech proficiency. Records of the grades 
for the students who participated in the project were forwarded to the 
researchers by the institutional contact some time after the 
administration of the posttests, (see Appendix A). 

*As in the case of the pretest the films shown in the posttests were 
arranged in three different orders with respect to viewing. This 
procedure allowed for each speech quality to be viewed at least once 
as the first, second, or third viewed speech by the student evaluators. 
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3.3 Techniques for Data Processing „ 

As data were received from the participating institutions they 
TA/oj-e checked thoroughly to make sure that the questionnaires and 
rating scales were correctly marked. In those cases where the 
participating institutions did not use a six digit student identification 
niimber one was assigned to the student data in such a manner that 
each was unique to one student and yet consistent across all data 
for the student. Data were then stored, with further processing 
deferred until that time when the grades for all students who 
participated in the project in the various courses at a specific 
institution were forwarded to the researchers by the institutional 

contacts . 

Beginning in July of 1967 data from the first eight replications 
of the project were submitted to the Michigan State University Office 
of Education Services for scoring by an IBM 1230 Optical Scanner. 

Output from this stage of processing was in the form of punched cards. 
These cards were then transformed by the MSU Control Data 
Corporation 3^00 computer under the control of a FORTRAN program 
especially developed for the project (see Appendix H) . Output from 
this stage of processing was again punched cards, but this time in 
a format suitable for analysis by the statistical procedures of the 

project. . ii 

As part. of the data processing stage of the project, all transformed 
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data were tabulated by the conlputer under the control of a program 
geared to examining the content of each data card. This procedure 
allowed the researcher to check for illegal characters in the data ^ 

caused by mis marked questionnaires and rating scales. This process 
also, allowed the researcher to examine the distribution of the data with 
respect to particular institutions. Here an attempt was made to 
determine"- whether the data from any particular institution were so 

i 

unique as to prevent inclusion in the final' analysis . 

Originally the plan of the project was to store all data on 
magnetic tape and then use such tape as the input medium for the 

statistical analysis developed for the proje/ct. However, as the . . 

data for the project were being processed, the Michigan State University 
Computer Laboratory was undergoing significant modifications of 
design and procedure which made it impractical to store the data of 
the project on magnetic tape. Thus, data remained stored on punched 
cards and these became the prime medium for input for statistical 
analysis. However, for many of the factor analyses of the project data 
were transformed from punched cards to a magnetic tape for further 
processing. The tape was released in each case after the appropriate 

analysis . Appendix I r^spresents the FORTRAN program used in the 

‘ 4 " 

transfer of data from punched cards to the magnetic tape. 
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3.4 Statistical Design of the Project 

The basic statistical procedure used for the project was. that 
of factor analysis. Specifically the research, was aimed at discussing 
the dimensions of speech evaluation as used and viewed by students 
of public speaking. Also of prime interest was die determination 
of the effects of training and proficiency in public speaking on the 
dimensionality of speech evaluation again in relation to students . 

The purpose of this article of the report it to consider the 

appropriateness of the statistical procedure of factor analysis for 

the intended research. The use of the technique of factor analysis 

as a means of determining common dimensions of evaluation is a 

relatively recent development in the field of Speech, but has often 

been used by psychologists in the area of measurement: 

Many psychologists have engaged in extensive 
testing programs, employing factor analysis to 
determine a relative small number of tests to 
describe the human mind as completely as 
possible. The usual approach includes the 
factor analysis of a large battery of tests in 
order to identify a few common factors . Then 
the tests which best measure these factors, or, 
preferably, revised tests based upon these, may 
be selected as direct measures of factors of the 
■ mind., (see, ref orence.3) ‘ 

The use of the techniques of factor analysis for derivation of 
items on a speech rating scale is exemplified by the work of Price 
at the University of Wisconsin. In his study, he used factor analysis 
as a method for examining the inter correlations among thirty-four 
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items contained on an experimental rating scale. It will be 
recalled from the discussion of the development of the rating scale 
for the project that the work of Price as well as that of Johnson, 

Lashbrook and Ralph was used in the development of the evaluative 
criteria that were used-on the speech evaluation '^scale. The reason 
for factor analyzing the results of student evaluation of the filmed 
speeches was to- determine the intercorrelation of the evaluative 
criteria with a view toward determining the effects of certain nominal 
variables on the dimensionality of the rating scale as developed. For 
the purpose of the research a dimension of speech evaluation was defined 
as a qluster of evaluative criteria highly correlated with one another 
and less correlated with clusters of criteria representing other 

i 

dimensions. Ideally, the items relating to one factor dimension should 
have a zero correlation with items representing another such dimension. 
With the interest of determining the solution which approaches this 
ideal a technique known as rotation is usually employed. It is the use 
of rotation that generally leads to reproducible factor structure or at 
least points to a legitimate method for comparing results from one 
factor analysis to another. Since such a comparison was essential 
to the research herein, a method of orthogonal. rotation^as employed 
for the purposes of interpreting data, (see reference 1) Via this 
process of rotation analysis the number of factors represented by the 
data could be determined. By maintaining a 90° angle for the 
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extractipn of the factors one can maximize the degree of association 
between criteria highly correlated with a given factor and minimize 
the association with factors representing other evaluative criteria. 
This measure of association is known as a factor loading. Thus 
for each extracted factor there is a factor loading for each 
evaluative criterion contained on the rating scale. As that loading 
approaches 1.00 the degree of association of the criterion with the 
factor (in terms of the cluster of items it represents) approaches the 

ideal. 

Employment of rotation in the interpretation of the results of 
factor analysis does demand the use of some objective criteria for 
determining when to stop extracting factors. This issue Is often 
phrased as the question "which solution is best to use? " For the 
purposes of determining the most- appropriate solution under the 
conditions of the study the following criteria were used. 

(1) For each factor identified there should be 
a minimum of two evaluative criteria with 

their highest loadings on that factor. 

(2) The solution chosen should maximize the 
number of evaluative criteria loading .50+ . 
on the factors represented by that solution. 

(3) The solution chosen should maximize the amount 
of variance accounted for within the framework 
of criteria 1 and 2. 



(4) The solution chosen should result in some 
type of meaningful comparison with solutions 
meeting criteria 1-3 but representing different 
levels of the research in terms of training 
and proficiency in public speaking. 

To the reader criterion number four might appear to be 

somewhat subjective . Guilford speaks directly to the rationale 

for the employment of such a criterion . 

In an unexplored domain where factors and their 
relations to tests are not yet known, lacking 
objective criteria, one may try out one hypothesis 
as to meaning after another until some solution 
seems sbtisfactory.- The injunction of meaning, 
even of Ihis hypothetical type , may lead to a 
final solution that also seems good objectively. 

(see reference 2) ' 

In addition to factor analysis the researchers were also 

interested in determining the effects of the prime variables of training 

and proficiency in public speaking on the reliability of the raters 

when viewing the filmed speeches. In order to determine the rating 

reliability of the students under the conditions of the research a 

technique known as the Intraclass Correlation was used. 

Many investigators seem to prefer the operation 
of correlating ratings obtained from different 
raters as the approach to reliability of ratings . 

There may be common bias among raters , but 
this source of error correlation is probably 
smaller than in re-ratings. One has to assume 
that raters involved in the reliability study are 
interchangeable. Since raters with similar types 
of information are generally used for this purpose, 
this assumption is not unreasonable, (see 
reference 2) 
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A special computer progra >ntitled CENTRA was employed 

! 

in the determination of the rating leliability of the students when 
viewing each set of films under the conditions of the project at 
each of the participating institutions, (see reference 5). 

Data coming from the student questionnaires were tabulated 
by the computer and used primarily to classify the student raters 

with respect to their previous training in public speaking. No 

( 

statistical analyses were performed on the student questionnaires. 
Results of the tabulations will, however, be reported in the following 

section of this report. 

The self- perception ratings given by the students in the 
pretest, posttestl, and advanced conditions of the research were 
also subjected to factor analysis . 

3 . 5 Summary of Section . ^ 

The following should be considered the prl ne objectives of 

the research: 

1. To determine specific dimensions of speech evaluation 
which can be used to make objective distinctions 
between varying qualities of public speaking. 

a. "What is the relationship between the 
dimensionality of the speech act and the 
type of scaling used to evaluate it? 

b. Will dimensionality vary significantly with 
a varying quality of speech presentation? 
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2 . To determine the nature of the relationship between ; 

training in speaking and the recognition and use 
of dimensions of speech evaluation which make 
distinctions between varying qualities of public 
speaking. 

a . Is there a difference between how students 
perceive the dimensionality of the speech 
evaluation before and after a basic college 
speech course? 

b. How much and what kinds of speech training 
are needed in order to broaden and/or refine 
a student's perception of the dimensionality 
of speech evaluation? 

3 . To determine the nature of the relationship between 
developed proficiency in public speaking and the 
ability to recognize and use objective dimensions 
of speech evaluation. 

a. Do good student speakers perceive the 
dimensionality of speech evaluation differently 
than do poorer student speakers? 

b. Do training in speaking and demonstrated 
proficiency in speaking make different types 
of contributions to how a student speaker per- 
ceives the dimensionality of speech evaluation? 

The basic data for the research were student evaluations of 
filmed speeches using a rating .scale developed specifically for the 
project. The films were also specifically developed for the project. 

The independent variables for the research were: (1) The 
stages of speech training represented by those students evaluating 
the filmed speeches; (2) The demonstrated speech proficiency represented 
by the students evaluating the filmed speeches as measured by their 
final grades in those courses used in the research (See Appendix A). 
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The dependent variables for the research were: (1) The 
number of identifiable dimensions (factors) used in the evaluation 
of the filmed speeches; (2) the consistency of factor structure 
(as represented by the factor loadings) determined via orthogonal 
rotation; (3) The percentage of rating variance accofihted for '.by the 
factors used in the evaluation of the filmed speeches; (4) The 
reliability of ratings for each evaluative criterion used in the 
evaluation of the filmed speeches . 
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TABLE 3.2-4 

Film Set Showings for Advanced Students and Actual Sample Sizes 
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RESULTS 

\ 



4.1 The Strategy for Reporting Results 

In developing a strategy for reporting the results of the research 

three rather significant decisions were made: 

(1) The results of the questionnaire would be used only 
to determine the possible descriptive differences 
between the beginning and advanced students who 
participated in the project. 

(2) All results would be limited to the reporting , in as 
objective a manner as possible, those types of 
statistical data most relevant to the basic design 
of the project (see Section III). 

(3) The factor analyses and reliability results as 
reported would always reflect the conditions 
under which the project data were ‘'actually 
collected (pretest, posttestl, posttest2, and 
advanced) .* 

The information sought via the questionnaire developed for the 
project related only to describing the course clientele who participated 
in the project. These course data came from either beginning speech 
students or advanced ones . Since there was no reason to suspect that 
course data of this type would change as a result of that course, the 
results of the questionnaire will be reported in two categories , beginning 
and advanced. The beginning category contains the questionnaire 
results for both the pretest and posttestl conditions under which 
data was gathered. The statistics reported will be limited to frequency 
and percentage of item response. 

*The exception to this being only the factor analyses of the self perception; 
in the posttest2 condition no such data were gathered, 
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It will b© r6m6mber6d that th© prim© int©nt of the research was 
to determine the effects of training and proficiency on the 
dimensionality of speech '^valuation. Dimensionality was defined 
in a factor analytic sense. An attempt was being made to discover 
the relationship of training and proficiency on: (1) the number of 
factors evolved for the act of speech evaluation; (2) the content of 
those factors in terms of the criteria represented on the speech 
rating form developed for the project; (3) the relative strength of the 
factors with respect to the percentage of scale variance which 
could be accounted for by the factor analytic solutions judged most 
appropriate. As supporting statistical data the researcher was 
interested, descriptively, in the abilities of the student evaluators 
to consistently maintain a distinction between the quality classifications 
of the filmed speeches used in the project. In order to provide some 
indication as to whether or not these classifications were maintained 
consistently, both the means and standard deviations established by 
the evaluators for each evaluative criterion contained on the rating 
form are examined. However, no attempt will be made to establish 
these classification categories beyond the descriptive level. Since 
the means and standard deviations yield information relative to the 
application of the statistical technique of factor analysis , they are 
reported prior to an examination of the various factor structures . 
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It is because the research is basically factor analytic that 
the decision was made to maintain the separation of data by conditions 
under which they were gathered. It will be remembered that with respect 
to training there was a correspondence of level of training to condition 
under which data were gathered. Training level 1 was represented as 
the pretest condition; Training level 2 as the posttestl condition; 
and Training level 3 as the advanced condition. The posttest 2 
condition was treated as an internal replication of the effects of Training 
level 2 on dimensionality. Proficiency is treated as a nested effect. 

It will be recalled that proficiency levels were appUed ex post facto 
to the data; that is, at the time of data gathering there was no 
attempt to establish the proficiency level of the evaluator. The point 
is that data were not gathered under the conditions represented by the 
combinations of training and proficiency levels. To act as though 
that were the case would confound the variances attributable to the 
main effects and work against the establishment of consistent 
factor structures. Since the essence of factor analytic study is 
based on the replication of factor structures, the results that follow 
will reflect the conditions under which the data were gathered. 

4.2 Administration and Analysis of the Questionnaire. 

As noted earlier, subjects in three conditions, pretest, posttestl 
and advanced completed the project questionnaire. For purposes of 
analyzing and interpreting the answers received, the data have been 
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classified according to course level. Therefore, participants 
are categorized as to "beginning" or "advanced" subjects. Table 
4^2-1 represents the frequency and percentage of response to each 
item contained on the questionnaire for the beginning and advanced 
students . 

4.3 Responses of the Beginning Students 

A sample size of approximately 3260 beginning speech students 
in two conditions — pretest and posttestl~~completed the questionnaire. 
A slight discrepancy between this figure and the number actually 
viewing the filmed speeches can be explained several ways. In 
some instances students entered the experiment section too late in 
the period to adequately complete the front side of the rating form 
before the filmed speeches were shown. In some instances students 
simply neglected to complete all questionnaire items. Finally, the 
disparity may exist as a result of mis-marked questionnaires that had 
to be discounted. 

Question one asked "Have you ever had a high school level 
public speaking course?" Respondents were instructed to answer 
"yes" or "no".' Of the 3180 students answering this question, 1193 
(37%) replied affirmatively, 1924 (61%) replied negatively and 63 (2%) 
of the participants chose to mark neither foil* It is difficult to ascribe 
meaningfulness to such figures in light of the absence of descriptive 
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data detailing the substance of the high school course. These 
data, however, could take on significant meaning if researchers 
in future study of the basic speech course attempt to study the 
correlation of previous course experience on performance in the 
basic course. 

Question two asked "Have you ever had a college level 
public speaking course?" A total of 685 (21%) had, in fact, 
completed a speech course on the college level prior to taking 
the basic course. Of that figure, 625_il9%) had taken the course 
at the same institution where they were enrolled in the basic course, 
the 60 (2%) had completed the course at another institution. The 
greatest number of responses and one to be expected, was the figure 
of those who had not completed a college level public speaking 
course prior to taking the basic course. That figure, 2240 (75%), 
would appear to reflect the fact that the basic course had no real 
prerequisite. A total of 135 failed to complete any of the three 
responses and comprised just 4% of the total responding to the second 
question. Interest should be paid the 21% who were enrolled in their 
second speech course . Future research might be directed at deter- 
mining the previous course content and the rationale for a second 
course. Thht is, if the beginning course is by definition the 
student's first exposure to speech, why then should it be the 



second course for many? 
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Question three asked those students who had taken a 

public speaking course prior to the beginning course to state 

the nature of the course. A total of 95 (3%) students had taken 

a course designated as a prerequisite; 703 (22%) had taken 

anpther course but it was not a prerequisite to the present 

course; and 2460 (75%) failed to respond, presumably because 

they had not taken a course prior to the ones in which they were 

currently enrolled. In a sense, the results of question three shed 

light on those of question two. That is , it might be safe to infer 

that the courses taken prior to the basic course in public speaking 

involved content not directly related and probably involved little or 

no platform experience for the student. Since platform experience, 

by tradition, is the feature of basic courses in public speaking and 

was the essential ingredient of those basic courses involved in the 

project, the results of questions two and three would not appear to 

require a subdividing of the training levels represented by the basic 

% 

course in public speaking. 

Question four asked, "Is this course the basic public speaking 
class offered at your university?" 2508 (77%) of those responding 
indicated it was, 480 (15%) answered negatively, and 262 (8%) did 
not know. Despite the results indicated by question four the researcher, 
in conjunction with the institutional contacts, determined prior to the 
gathering of data that the courses representing the beginning training 
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level were in fact the basic courses in public speaking at the 

participating schools. Question four then reflects on the part of some 

students a lack of knowledge with respect to the courses in which 
*• 

they are enrolled. 

Question five — "Does this course have basic public 
speaking as a prerequisite? "—142 (4%) of those participating 
indicated "yes", 2930 (90%) said "no", and 187 (6%) did not 
know. Again, prior to the gathering of data the researcher 
determined that the courses at the beginning level involved in the 
project did not have a prerequisite involving public speaking. 

Question six attempted to establish student identification 
of the department offering the basic course. Of the 3259 responding, 

128 (4%) identified the basic course as being offered in a Communication 
department, 182 (6%) placed the basic course in the English department, 
2684 (82%) identified the course as one of the Speech department, 

168 (5%) recognized the course as being in some "other" department, 
and 97 (3%) did not know which department offered the course. 

Question 7 asked the students to grade themselves as a public 
speaker. 166 (5%) graded themselves as A, 1282 (39%) as B, 1428 
(44%) as C, 221 (7%) as D, 23 (1%) as F, and 139 (4%) preferred to 
record no grade. The mean grade of this group represented essentially 
C+ grade level (x=2.43). 
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Question eight asked respondents for their class standing. 

As expected, the majority of students enrolled in the beginning 
course were first and second-year students. The overall distribution 
was: Freshman; 982 (30%); Sophomores: 1021 (31%); Juniors: 709 
(22%); Seniors 451 (14%); Graduates: 10 (.3%), and other: 86 (2.7). 

Question nine asked participants to identify their majors. 

The options offered were all speech related and nearly all responding — 
2995 (92%) — were hot majors in one of the options. The remaining 
breakdown was: Radio and Television - 90 (3%); Rhetoric and Public 
Address - 9 (.3%); Speech Education - 95 (3%); Speech Science - 26 
(.7%); and Theatre - 44 (1%). This evidence tends to support the notion 
that the basic course in public speaking is primarily a service function 
offered by Speech Departments to other disciplines within their 
respective institutions . 

In summary the typical participant (in the beginning courses) 
who was involved in the research project would be a freshman or 
sophomore without a prior speech course, cognizant of the department 
offering the course and representing a non-speech major with a C+ 
grade perception of himself as a public speaker. 

4.4 Responses of the Advanced Students 

All participants in the advanced condition completed the same 
questionnaire as those in the beginning condition. Inasmuch as the 
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questions have been detailed in the previous section, only the 
specific responses will be discussed here. 

In question one, over half of the 547 participants who 
completed the question (291-53%) had taken a high school level 
public speaking course. This percentage is considerably higher 
than those beginning students who had had similar experience 
(37%) . The advanced students who had not ever had a high school 
course totaled 241 (46%), and 5 (1%) questionnaires could not be used. 

The number of advanced students who had completed a college 

level course either at the present institution or at another totaled 
* 

468 (85%), 50 students (9%) did not have a prior college level course 
and 29 questionnaires (6%) could not be used. Many reasons can be 
offered for the fact that some did not enroll in a prior course . Year 
in school, proficiency, high school experience, or even quality of 
the advanced course provide several explanations. Most institutions 
which participated in the project did allow some form of advanced 
placement of students (generally involving the waiver of any 
prerequisite requirements) . 

In answer to question three asking forthe nature of the prior 
speech course, 389 (71%) indicated that the prior course was a 
prerequisite, 63 (11%) had taken a prior speech course, but one 
that was not classed as a prerequisite, and 95 (18%) failed to respond 
to one of the noted options; presumably many of this number had not 
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had a public speaking course prior to the one they were enrolled ^ 
in. Prior to the gathering of data the researcher had determined 
that the advanced courses used in the project did have a prerequisite 
of basic public speaking. 

Question four asked the students whether or not the course they 
were taking was the basic course at that school. One would expect 
that nearly all would reply negatively. In fact, 74 (14%) said it was, 
460 (84%) acknowledged that it was not the basic course, and 13 
(2%) did not know. 

Question five provided data that appear to be in conflict, 
at least partially, with the answers to number four. In question five, 
488 (89%) indicated that the course had as a prerequisite a basic 
public speaking class, 47 (9%) said no, and 12 (2%) did not know. 

The question arises as to how a course can be the basic public 
speaking course at a university (question four) and still require 
basic public speaking as a prerequisite (question five). 

The advanced students appeared to be as perceptive in 
identifying the advanced course with the speech department, 459 
(84%) acknowledging such identification. Interestingly enough, 
a higher percentage of the advanced students identified the 
advanced course with the English department (9%) than did the 
beginning students (6%) , 
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Question seven, asking for a self appraisal as a public 

speaker in terms of a grade, revealed a considerably higher mean 

arade noint estimate than did the beginning students (advanced: 

* 

x=3.06; beginning: x=2.43), with the following total distribution: 

A, 107 (19%); B, 356 (65%); C, 60 (11%); D, 5 (1%); F, 1 (1%), and 
18 (3%) preferred to record no grade. 

As would be expected, the class standing of the advanced 
student was higher, the majority of enrollees being juniors and 
seniors, 373 (68%). The total distribution was: Freshman; 33 (6%); 
Sophomores: 121 (22%); Juniors: 212 (39%); Seniors: 161 (29%); 

Graduate: 8 (2%); and other: 12 (2%). 

Only 29% of the advanced participants were speech-related 
majors. The following was the total distribution: Radio and television - 
46 (8%); Rhetoric and Public Address - 25 (4%); Speech Education - 
58 (11%); Speech Science - 10 (2%); Theatre - 21 (4%); and other - 387 
(71%). This was, however, a considerably higher percentage of speech 
related majors than was found among the basic students . 

In summary the typical participant (in the advanced courses) who 
was involved in the research project would be a junior or senior 
with prior speech experience (probably in the form of a basic course 
in public speaking) , cognizant of the department offering the course k 
and representing a non-speech major with a B grade perception of 
himself as a public speaker. 
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4.5 Factor Analysis of the Self- Perception Scales 

The students in the pretest, posttestl, and advanced 
conditions of the research filled out a self-perception rating form 
at the same time that they responded to the questionnaire . The 
self-perception scales were similar to those to be used in the evaluation 
of the filmed speeches. The difference in the self-perception scales 
was that the criteria were rephrased in order to be applicable to the 
evaluator rather than to a speech being evaluated. There were 
essentially two reasons for having the students fill out the self- 
perception scales: (1) it was felt that asking the evaluators to apply 
the rating scales to a known speaker would faj^iliarize them with both 
the content and the format of the rating instrument; (2) it was felt 
that a factor analysis of the self-perceptions would be a gauge to 
the actual dimensionality of the rating instrument. 

Table 4.5-1 represents the means and standard deviations for 
the self-perception scales under the three conditions of the research 
in which they were completed. In general, it can be stated that with 
increased training in public speaking there is a corresponding increase 
in the mean self-perception ratings of students . The most marked 
increase seems to occur as a result of the basic speech course. There 
is no consistent shifting in the scale variances that can be attributed 
to training in public speaking. These findings largely parallel those 
of the pilot study (Table 2.4-1) with respect to beginning and 
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advanced students at approximately the middle of their respective 
courses. 

Tables 4.5-2, 4.5-3, and 4.5-4 represent the factor analyses 
for the self- perception ratings for the pretest, posttestl, and advanced 
conditions of the research respectively. It will be noted that in each 
instance a four factor solution was deemed mos: appropriate. This 
represented a slight change from the pilot st^dy data where for the 
advanced students a five factor solution was evolved. Across the 
conditions under which self-perception data were gathered there was 
agreement with respect to factor content on 12 of the 19 scale items. 
This agreement is reflected by the high factor loadings of evidence . 
logical reasoning , and organization on one factor; the items diction, 
fluency . vividness , vocal inflection and word choice on a second 
factor; the items attitude . enthusiasm and interest representing a 
third factor; and finally the item physical appearance associated with 
a fourth factor. When comparing the factor content between the 
posttestl and advanced conditions we find agreement on 16 of the 19 
scale items . The exceptions are facial expression , imagination and 
preparation . In comparing the pretest self- perception content to that 
of posttestl we see agreement on the additional item imagination . 

Thus , it would appear that with respect to the factor content of their 
self- perceptions the students completing the basic speech course 
moved closer to the self-perceptions of the advanced students 
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and away from those who had yet to complete the basic course. 

There is no consistent indication of a significant effect on factor 
content of the self-perception data attributable to training in public 
speaking beyond the basic level. 

With respect to the total amount of accountable variance there 
appears to be a slight increase for the advanced students over the 
basic ones regardless of training level. This increase was certainly 
below that obtained for the pilot study data (see tables 2.4-2 and 
2.4-3). There appears to be no consistent finding with respect to 
the relative strength of the factors between solutions. 

Two conclusions would seem to be indicated with respect 
to the analyses of the student self-perception scales. First, as in 
the case of the pilot study, the students appear to use the rating form 
as a multi-dimensional instrument. Second, there is some evidence, 
particularly with respect to factor content, to suggest differences in 
factor structure due to training in public speaking, or at least that 
training represented by the basic courses involved in the project. 

4.6 The Effects of Film Quality on Factor Structure 

It will be remembered that the chief variables under investigation 
for this report were those of training and proficiency in public speaking. 
Of secondary interest was the determination of the effect of varying 
filmed speech quality on the factor structure of the rating form 
developed for the project. In order to gauge this effect the data 

er|c 



IV-15 



stemming from all the conditions under which data were gathered 
were subjected to factor analyses with the classification variable 
being film quality. That is, data from each film quality (low, 
middle, high) for each condition of the research (pretest, posttestl, 
posttest2, and advanced) were subjected to analysis. It is important 
to note that the factor structures represented in this article are those 
which distinguish between film qualities, but do not represent the 
factor structures evolved from student evaluators attempting to make 
a distinction between filmed speech qualities. It is for this later 
circumstance that the effects of the main variables were desired. 

4.7 Factor Structure for Film Qualities in the Pretest Condition 

of the Research 

Table 4.7-1 represents the means and standard deviations 
for the three film qualities as evaluated in the pretest condition 
of the research. One can easily see that the mean ratings assigned 
in the pretest reflect the three filmed speech classifications. This 
is a consistent finding across the 19 evaluative criteria contained 
on the speech rating form. With the exception of the criterion 
physical appearance the range with respect to standard deviations 
across the film classifications is two tenths of 'i point or less. 

The?e findings then were not viewed as preventing a comparison of 
factor structures evolved for the quality groups in the pretest condition. 
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Tables 4.7-2, 4.7-3, and 4.7-4 represent four factor 
solutions for each film classification as viewed in the pretest 
condition. In all instances it was the four factor solution which 
adhered to the criteria cited in Section III for interpreting factor 
analyses. One can see emerging for each film quality a dimension 
representing the criteria of total effect, evidence . logical reasoning . 
organization and preparation . A second factor reveals the clustering 
of the criteria diction . fluency and word choice . The third factor 
reflects the items physical appearance and poise. The fourth and 
final factor across the film qualities involved the scale items of 
attitude . enthusiasm and imagination. There does appear to be 
agreement between the low and middle factor structures on the 
additional evaluative criteria of vividness and vocal inflection, 
interest , and eve c ontact. Furthermore the item facial expression 
reflects a parallel between the middle and high speeches. The item 
bodily movement i ndicates no degree of consistency across filmed 
speech qualities. 

In examining the total percentage of scale variance accounted 
for by the four factor solutions across the film classifications we can 
see that the evaluations of the high quality film presentations accounted 
for more than the other two which did not differ in regard to this measure. 

In using the contribution of common factor content to the total 




amount of accountable variance one can see a slight difference between 
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i film classifications in terms of the ranking of dimensions. With 

respect to the total effect, evidence . logical reasoning . 
r organization and preoaration factor it Vvas for the middle and high 

I classifications that the most variance was accounted for. For the 

low and middle classifications the factor representing attitude , 
enthusiasm and imagination accounted for the third highest amount 
of scale variance. The diction . fluency a nd word choice dimension 
: shifted in importance with respect to the film classifications, ranking 

first for the low films and third for the high. Finally, the physical 
appearance and poise dimension in all cases accounted for the lowest 
percentage of the scale yariance in the pretest condition. 

I 

I 4.8 Factor. Structure for Film Qualities in the Posttestl Condition 

of the Research 

!i> 

f Table 4.8-1 represents the means and standard deyia.tions for 

I the three film qualities as eyaluated in the posttestl condition of the 

■ research. The findings indicate very little difference from those 

discoyered in the pretest condition, namely, that the mean ratings 

t 

, consistently parallel the film classifications and that there is very 

little variance among the criteria for a given classification and 
between classifications across the 19 evaluative criteria. 

Tables 4,8-2, 4.8-3, and 4.8-4 represent those factor analytic 
solutions which seemed most appropriate for the film classlfiCcitions 
as viewed in the pc jttestl condition. It will be noted that four factors 

r 
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consistently emerged. There appears to be common clustering on 
13 of 19 evaluative criteria. It can be seen that the scale items 
evidence , logical reasoning , organization and preparation 
represent a single factor for all quality groupings. This is 
also the case for diction , fluency and word choice as well as 
physical appearance and poise . A fourth dimension is represented 
by attitude , enthusiasm, facial expression a nd interest . For the 
remaining six eyaluatiye criteria there are parallel loadings for 
total effect, bodily moyement and vocal inflection for the low and 
high filmed speech classifications. For the middle and high 
categories this parallelism exists for the items imagination and 
vividness , The scale criterion eve contact shows a common factor 
association for the low and middle quality films. 

As was the case with the pretest factor analyses there appears 
to be a higher percentage of accountable variance for the high film 
classification than that for the low and middle types of filmed speeches. 
In terms of ranking the factors across the film qualities it was found 
that there was no unanimity. The strongest factor across the film 
qualities was the one involving evidence . logical reasoning , 
organization and preparation . The weakest factor appears to be the one 
involving physical appearance and poise. For the other two factors 
there appears to be no consistency with respect to the contributions to 




the total amount of accountable variance . 
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4.9 Factor Structure for Film Qualities in the Posttest2 

Condition of the Research. 

It will be remembered from the discussion of the logistics 
of the project that posttest2 was designed as a replication of 
posttestl with the exception that in the former case the data 
stem from students who participated in the pretest but viewed a 
different set of films at the individual institutions . 

Table 4.9-1 represents the means and standard deviations for 
the three film qualities as evaluated in the posttest2 condition 
of the research. As was the case in the pretest and posttestl 
conditions, the mean ratings consistently reflect the differing film 
qualities . All the standard deviations seem to fall within the ranges 
observed for the evaluative criteria in the pretest and posttestl 
conditions. 

Tables 4.9-2, 4 .9-3, and 4.9-4 represent the factor structures 
for the film classifications for data stemming from the posttest2 
condition. With the exception of the items poise and imagination 
the results are the same as discovered for the posttestl condition. 

Again four factor solutions were judged most appropriate with 
evidence , logical reasoning , organization and preparation clustering 
on one factor and diction, fluency and word choice another dimension. 

In the posttest2 condition imagination linked with attitude . enthusiasm, . 
facial expression and interest to form a separate factor. The item 



physical appearance stood out as representing a fourth factor across 




the quality groupings. Also, as was observed in posttestl the 
total effect loaded in similar fashion for the lo\. and high 
classifications and vividness a ssociated itself with the same 
factor content for the middle and high films . It was only for the 
low classification that the item poise did not align itself with 
physical appearance. Unique to the posttest2 condition were 
the facts that the item of bodily movement l oaded in a similar fashion 
for the middle and high classifications, and that vocal inflection 
(as in the case of the pretest) paralleled the low and middle film 
qualities. The item eve contact indicates no degree of consistency 
across the filmed speech categories. 

With respect to the total amount of accountable variance there 
seems no meaningful difference between the factor analyses (though 
there is a slight dip in the total for the middle speech). With the 
exception of reversing of ranks for the diction, fluency and word 
choice factor and the physical appearance and poise factor there 
is no difference between the posttestl and posttest2 conditions. 

The evidence cited in this article seems to indicate that the 
use of the rating forms and the viewing of a second set of films 
by students in posttest2 does not constitute a significant degree 
of difference attributable to familiarity with the project research 
materials. The discovery of any such effect was the purpose of the 
internal replication represented by posttest2. 



4,10 Factor Structure for Film Qualities in the Advanced 

Condition of the Research. 

Table 4.10-1 represents the means and the standard deviations 
for the three film classifications as evaluated in the advanced condition 
of the research. As in the case of the other research conditions/ it 
can be seen that the mean ratings for the 19 evaluative criteria 
contained on the rating form consistently maintain the film 
classifications. The standard deviations on the other hand do not 
consistently vary between speeches nor between criteria. 

Tables 4.10-2/ 4.10-3/ and 4.10-4 represent the factor 
analyses -for the film classifications as viewed in the advanced 
condition of the research. It will be noted that for the low speeches 
a five factor solution was judged most appropriate. For the other 
two quality categories a four factor solution was most acceptable 
according to the criteria developed in Section III of this report. This 
finding seems unique to the advanced condition of the research. With 
respect to the content of the identifiable dimensions there appears 
to be less agreement than that observed with the other conditions. 

Only 9 of the 19 scale items appeared to hold across film qualities. 
There is clearly established a common factor involving the items 
evidence , logical reasoning , organization and preparation . Also 
emerging is a dimension relating attitude . enthusiasm and interest. 
There iS apparently an identifiable factor involving the items poise and 
physical appearance. Missing across the three film classifications 
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for the advanced condition was the factor represented by diction . 
fluency and word choice. The agreement of factor content picks up 
considerably when comparing the factor structures evolved for the low 
and middle speeches in the advanced condition. Here the items of 
diction . fluency and word choice d id emerge as a separate factor. In 
addition the items of bodily movement a nd vividness a ssociate with a 
common factor for this same comparison. By comparing the other 
combinations of factor structure s'^Cldw to high and middle to high) we 
see in each case the loading of one additional item in a similar fashion 
to those 9 which extend across all the film categories. For both the 
low and high speeches the item total effect clusters with evidence . 
logical reasoning , organization and preparation to form an identifiable 
dimension of speech evaluation. In the cases of the middle and high 
classifications vocal inflection lines up with attitude , enthusiasm and 
interest to form a separate factor. Thus, with respect to the factor structures 
across the film qualities for the advanced condition we find an additional 
factor emerging for the evaluation of the low speeches, but one that seems 
to have only a slight effect on the establishment of a common factor 
structures. That is, the fifth factor seemed to be composed of items not 
found in previous instances to be a factor (the items facial expression and 
eve contact) . but the emergence of this factor did not destroy the 
content of the dimensions discovered under the previous conditions 
of the research. By far the most significant influence on factor content 
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was the advanced students' evaluation of the high film classification. 
But, even here the four factors discovered previously did hold up — the 
difference being primarily limited to one or two items loading on a 
secondary factor rather than a primary one as noted previously. 

With respect to accountable variance there appears to be very 
little difference among the film classifications. There is a slight 
increase in the total amount of variance accounted for in the evaluation 
of the high quality filmed speech. This finding was also observed 
for some of the previous conditions of the research. With respect 
to the rankings of the individual factors according to their contribution 
to the total scale variance we see much agreement, with the most 
significant factor being the evidence-logical reasoninq-organizationr 
preparation dimension and the least the physical appearance -poise 
factor. The remaining factors seem to shift in importance with the 
changes in film classifications. 

4.11 Summary of Finding with Respect to the Factor Structures of 
the Various Film Qualities. 

Two conclusions seem to emerge from an examination of the 
factor structures for the film classifications across the conditions of 
the research. First, the evidence indicates that in each condition of 
the research the film classifications of low, middle and high were 
maintained. Second, that despite some slight discrepancy for the 
individual film classifications within particular conditions under which 
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data were gathered, a common four factor solution seemed to 
emerge. Above all, and most importantly for determining the 
effects of the main variables of training and proficiency, the 
rating scale remained multi-dimensional in all conditions of the 
research across all film qualities. 

4.12 The Effects of Training in Public Speaking on the 

Dimensionality of Speech Evaluation. 

In an attempt to determine the effect of training in public 
speaking on the dimensionality of speech evaluation the data 
processed for the study were first analyzed according to the 
conditions under which they were gathered. The conditions 
themselves represent three training levels: U) evaluation of 
filmed speeches by students enrolled in a basic course in public 
speaking in the first weeks of that course (the pretest condition); 

(2) evaluation of filmed speeches by students enrolled in a basic course 
in public speaking during the final weeks of that course (the posttest 
condition); (3) evaluation of filmed speeches by students enrolled in 
advanced courses in public speaking during the final weeks of that 
course (the advanced condition) . It will be remembered that the 
posttest condition was further subdivided into posttestl and posttest2 
in order to allow for an internal replication of the effects of the 
basic course (see article 3.2). 

In examining the effects of training in public speaking on the 
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dimensionality of speech evaluation each student's ratings for all 
3 film classifications were used. This means that for the purposes 
of analysis each student provided three evaluations (of a low, a middle, 
and a high speech ). At this point the researcher actually began to 
tackle th main issue; the focus on the emergence of the factor 
structures which are used by students at various training levels 
discriminating among filmed speeches shown in a series. 

4.13 The Pretest Condition - Training Level 1 

Data collected under the pretest condition were from students 
at the start of a basic course in public speaking. An attempt was made 
to gather all thes •' data during the first week of classes. In the 
instances where this was impossible the data were collected during 

the second week of classes but prior to assignments involving practice 
in public speaking. 

Table 4.13-1 contains the means and standard deviations for the 
19 evaluative criteria contained on the rating scale. The means ranged 
from a high of 4.93 on the scale item physical appearancfi tn a low of 
3.80 on the scale item fluency . The standard deviations fell between 
for attitude^ to 1.7 for organization. These findings did no+ am to 
necessitate any adjustment in scale values prior to the employment of 
factor analysis . 
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Table 4.13-2 represents the results of the factor analyses of the 
pretest data . A three factor solution was judged to be most appropriate 
for interpreting the pretest data . The first factor extracted represented 
the items total effect, attitude, bodily movement, diction , enthusias jL 
faoial ovnression . fluencv.. imagination, interest ^ vlvldnes s_and 
T,ooal inflection. The second factor combined the items evidenc e 
looioal reasoning , organization and preparation . The third factor 
included eve contact , nhvslcal appearance and polse _ . 

It can also be seen from Table 4.13-2 that the three factor 
solution for the pretest data accounted for approximately 70% of the 
total scale variance. 

4.14 The Posttest Condition - Training Level 2 

Data collected under the posttest conditions were for students 
-.at the conclusion of a basic course in public speaking. It will be 
remembered that in the posttest condition data were collected on 
film evaluations for students who both did and did not participate 

in the pretest condition. 

Table 4.13-1 contains the means and standard deviations for 
those students who did not participate in the pretest condition on the 
19 evaluative criteria contained on the rating scale. The means 
ranged from a high of 4.98 on physical appearanc e. to a low of 3.80 
on vividness. The standard deviations fell within the range of 1.3 
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for attitude , facial expression and physical aopearannfi to 1. 6 for 
evidence , eye contact , logical reasoning , organization and preparation. 
Table 4.13-1 also contains the means and standard deviations for the 
students who did participate in the pretest on the 19 evaluative criteria 
contained on the rating scale. The means ranged from a high of 4.85 
on physical appearance to a low of 3.83 on imagination . The standard 
deviations fell within the range of 1.2 for physical appearance to 1.6 
for organization . Again the differences in means were not such as to 
necessitate any adjustment in scale values prior to factor analysis. 
Furthermore, the differences between means and standard deviations 
for the 19 criteria for the pretest and posttest conditions were not 
viewed as sufficient to prevent meaningful comparisons between 
respective factor analyses. 

Tables 4.14-1 and 4.14-2 represent the factor analyses for 
the two posttest conditions. It will be noted that both represent 
four factor solutions, and that v/ith respect to the clustering of 
items about the four factors there is agreement between the two posttest 
conditions on 18 of the 19 scale items. The exception is the criterion 
bodily movement. The high degree of similarity between the factor 
structures of posttestl and posttest2 does not seem to support a 
conclusion that the repeated use of the rating scale represented a 
significant degree of training for those students who participated in 
both the pretest and posttest conditions. In comparing the posttest 
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conditions to that of the pretest we see a degree of similarity that 
supports three common factors. In all three conditions the items 
total effect, attitude . enthusiasm, facial expression , imagination . 
interest , vividness a nd vocal inflection represented the first 
extracted factor. The items evidence . logical reasoning , organization 
and preparation represent the second extracted factor. The items physical 
appearance and poise represent the third extracted factor. With respect 
to the posttest conditions we can see in both cases the emergence of a 
fourth factor involving the criteria diction , fluency a nd word choice. 

Thus it would appear that the contribution of a basic course in public 
speaking to the dimensionality of speech evaluation (when that evaluation 
involves making a distinction between qualities of public address) 
is to the use of a language facility factor involving the criteria diction , 
fluency and word choice . 

In terms of accountable variance we can see very little difference 
between the two posttest conditions either in the total percentage of 
variance accounted for or the rank order of the respective factors in 
terms of their contributions to this total. In comparing the posttest 
conditions to the pretest we see that the extraction of the dimension 
of diction . fluency and word choice allows the students completing 
a basic course in public speaking to increase the total percentage 
of scale variance when discriminating between the qualities of the 
filmed speeches. 
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4.15 The Advanced Condition - Training Level 3 

Data collected under the advanced condition were for 
students enrolled in various courses in public speaking which 
U) had the prerequisite requirement of a basic course in public 
speaking; and (2) involved training and practice in public speaking.* 
Table 4.13-1 contains the means and standard deviations 
for students who participated in the advanced condition of the 
research. It will be remembered that these students viewed a particular 
film set at the conclusion of their respective courses, that the film 
set evaluation was the same as that used in the pretest and posttestl 
condition of the research, and that all film sets were represented in 
the data collected in the advanced condition. The means for the 
advanced students ranged from a high of 4.76 on physical appearance 
to a low of 3.37 on imaqina tion . The standard deviations fell within 
the range of 1. 3 for total effect to 1 , 6 for organization and evidence. 

It will also be noted that the means for the advanced conditions are 
consistently lower than for the previously described training levels. 

This is despite the fact that there is no corresponding increase or 
decrease in the standard deviations. This difference in means alone 
is not sufficient to prevent a comparison between factor structures 

*As data were received from each participating institution, a check 
was made to determine (1) whether there were any simultaneous en- 
rollments in advanced and beginning courses; (2) that no student 
participated in the research more times than the design allowed. In 
instances where this did occur the student's data were dropped 
from these analyses. 
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ficross trsining l6v©ls « Nor wor© th© diff©r©nc©s on individu©! 
criteria within th© advanc©d condition de©m©d sufficient to 
necessitate any adjustment in scale values prior to factor analysis . 

Table 4.15-1 represents the factor analysis for all students 
providing data in the advanced condition of the research. For the 
purposes of a meaningful comparison between training levels (note 
the mention of this criterion in Section III of this report) a four 
factor solution was chosen. In comparing the factor structure of 
the advanced students to the four-factor solutions of the students 
who provided data under the posttest conditions, we can note 
agreement on 18 of the 19 scale items for the combination advanced- 
posttestl (same film set), and agreement on 17 of the 19 items for 
the combination advanced-posttest2 (different film sets). The 
non-consistent items across these combinations are bodily movemen t 
and eve contact . The item bodily movement_w as the inconsistent 
item across all three conditions under which data were gathered. 

In comparing factor structures across training levels (pretest, 
posttest, advanced) one can see agreement on the following items 
with respect to identifiable factors. The first extracted factor is 
represented by total effect, attitude, enthusiasm, facial expression, 
interest . imagination , vividness a nd vocal in flection. The second 
extracted factor is represented by evidence , logical reasoning, 
organization and preparation. A third factor across training levels 
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is represented by the items physical appearance and poise. For 



the students who completed a course in public speaking (posttestl 
and posttest2) ^ as well as those completing a course in advanced 
public speaking, a fourth factor of evaluation emerged. That factor 
is represented consistently by the items diction . fluency and word 
choice . 

While a four factor solution for the students in the advanced 
courses allowed for the most reasonable comparisons across training 
levels^ it was also true that a five factor solution did meet other 
criteria for an acceptable analysis. Table 4.15-2 represents the 
five factor solutions for data collected under the advanced condition. 

It will be noted that the extraction of a fifth factor seems to diminish 
the additive effects of the previously cited factor structures for the 
different training levels. That is, with the five factor solution 
only the evidence -logical reasoninq-etc. and the physical appearance- 
poise factor remain intact. The fact that a five factor solution does 
meet some of the requirements for an acceptable factor analysis does 
allow the construction of an interesting hypothesis with respect to 
advanced training in public speaking, namely, that courses differing in 
content may well have a different effect on the dimensionality of speech 
evaluation. An examination of the data collected under the advanced 
condition revealed that the data came from a wide variety of courses 



and that any attempt to study the differential effect of the varied course 
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contents would reduce the sample to such small subgroupings as 
to make factor analysis unwarranted. Thus a decision was made to 
allov/ the four factor solution to stand for the advanced data, 
recognizing that it would prevent any meaningful statement to be 
made with respect to the contribution of this training level to the 
dimensionality of speech evaluation. 

In terms of accountable variance we see that the four factor 
solution for the advanced students accounts for less of the total scale 
variance than similar solutions for students completing a beginning 
course in public speaking. In fact, the accountable variance seems 
very close to the amount provided by beginning students prior to taking 
the basic course. Two rationales might provide an explanation for this 
finding. The first would be a lower degree of reliability for the advanced 
students , and the second a differential effect attributed to the wide 
variety of course content at the advanced level of training. It is 
interesiing to note that the five factor solution for the advanced data 
adds less than three percent to the total accountable variance, which 
is still below that of the four factor solutions for students completing 
a basic public speaking course. This finding tends to support the 
rationale dealing with reduced reliability for the students in the 
advanced courses. The issue of reliability will be covered in a later 
division of this section of the report. ‘ In terms of the rank ordering 
of the factors it can be seen that for both posttests and the advanced 
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factor analyses the rankings are the same. 

4.16 Scale Item Reliability Across the Training Levels. 

It will be recalled that the method to be used to determine 
the reliability of the rating scale items was that of the Intraclass 
Correlation. The coefficients deemed most meaningful given the factor 
analytic design of the project were those dependent upon the combined 
ratings given by viewers over the three films seen at a particular 
showing. The statistical technique of the Intraclass Correlation 
was performed on all data representing a unique film showing. It 
will be remembered that four film sets were used in the research 
and further that within a given film set the films were shown in 
three distinct orders. This means that within each condition of the 
research at least 12 reliability coefficients were computed for each 
criterion contained on the rating scale. In actuality reliability 
estimates were run on each film showing even if it represented a 
replication of a particular combination of film set and order. Since 
neither number of showings or order of film arrangement was a variable 
under study, it was decided that the reliability of each film set on 
each criterion would be estimated by taking the mean combined intraclass 
correlation coefficients for the various combinations of order and 
showing. This meant that any variance in reliability due to film 
order or replication of film order for a given film set was dropped 
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from the consideration of reliability.* 

Table 4 ..16-1 represents the Intraclass Correlation coefficients 
for the 13 scale items across the training levels. In each instance 
the correlations represent means across the four sets of films shown 
in each condition under which data were gathered.** In order to gauge 
the possible effects of training on the reliability of the scale items, 
the many Intraclass Correlation coefficients were subjected to an 
analysis of variance. The approach used treated both training level 
and scale item as main effects . In addition an interaction of training 
level and scale item was also extracted. All other factors (film- set, 
order, showing) which could contribute to variance were relegated to 
the error factor. A decision was made to treat the posttestl condition 
as representative of training level two. The analysis revealed a 
significant difference in reliability due to training ( ^ .0005). There 
was also a difference in the reliability due to the effect of the 
• scale items (.035) . However, there was no significant interaction 
between training level and scale items. Further analyses revealed 
that generally the reliability established at the end of the course in 

*Also cropped from consideration in this final report was any effect 
on reliability due to particular institutions, instructors, and courses 
within conditions under which data were gathered . 

**It should be remembered that training level 2 (completion of a basic 
course in public speaking) was represented by two conditions under 
which data were collected: posttestl and posttest2. 
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public speaking was slightly higher across the nineteen evaluative 
rating scale criteria than that established for the beginning 
speech students prior to taking the basic course in public speaking. 
Also it was found that the reliability for the students in the advanced 
courses was considerably below that of the beginning students regard- 
less of training level across the 19 scale items.* It is interesting 

to note that the reduction in reliability for the advanced students might 

« 

well be sufficient to explain the corresponding lessening of the four 
factor structure with respect to accountable variance when compared 
to the posttestl and posttest2 conditions of the research. 

4.17 Effects of Proficiency in Public Speaking on the Dimensionality 
of Speech Evaluation. 

In an attempt to determine the effect of proficiency in public 
speaking on the dimensionality of speech evaluation the data from 
the research conditions were divided according to the final grades 
received by the students in their respective courses. Table 4.17-1 
represents the grade distributions for students who participated in 
the project under the various research conditions. 



*As a point of interest the same analysis with respect to the Intraclass 
Correlation coefficients was run with the posttest 2 condition representing 
the second training level. The results were essentially the same as 
described above with both training (.022) and scale item (.012) having 
an effect on reliability but not the interaction of the two. There was a 
greater correspondence between the reliabilities for the pretest and 
posttest2 condition, with most of the effect of training being due to 
the consistently lower coefficients established for the advanced students. 
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The data for each student receiving a particular grade under the 
conditions of the research consisted of his evaluations of the three 
fiLned speech presentations according to the ni leteen criteria contained 
on the speech rating scale. Tables 4.17-2 to 4.17-5 represent the 
means and standard deviations for the four research conditions on 
the nineteen criteria for each grade level. These results indicate 
very little differ.. nee between the pretest and the posttests means and 
standard deviations for those students enrolled in basic courses 
in public speaking across the five grade levels. However, it is 
interesting to note that the advanced students consistently, across 
the five grade levels , had lower means on the criteria for the filmed 
speeches than the students in the beginning courses. This was despite 
the fact that there was no consistent difference in the standard deviations 
that could be attributed to either the research condition or proficiency. 

In order to seek out possible difference in factor structure 
which could be attributed to proficiency in public speaking, the 
data for each condition were divided according to the grades received 
by the student evaluators. Each of these new divisions was subjected 
to factor analysis with the criteria cited in Section III used in 
determining the appropriate solutio.n. 

4.18 The Pretest Analyses for Proficiency Levels. 

It should be noted from the outset that in determining the 
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effects of proficiency on the factor structure of the pretest data, 
the measure used for division was still the final grade received 
by the student for the courses in which the data were collected. 

This makes the determination of the effects of proficiency in public 
speaking on the dimensionality of speech evaluation in the pretest an 
ex post facto investigation. Also to be considered is the fact that 
the pretest data stern only from students enrolled in beginning speech 
courses. 

Tables 4.18-1 through 4.18-5 represent the pretest factor analyses 
for A, B, C, D, and F students respectively.* In terms of the number 
of identifiable factors for the five grade levels it can be seen that 
for the A students a four factor solution seemed most appropriate, 
while for the B through F students a three factor solution was the 
most meaningful. 

In terms of the content of the factors it ca:<i be seen that for 
all grade levels the evaluative criteria of evidence . logical reasoning , 
organization and preparation all loaded highest on the same factor. 

This same relationship held for the evaluative criteria of imagination 
and interest. With the exception of the F students the evaluative 



*The number of Pretest students who completed a course in public 
speaking was somewhat less than the number who participated in the 
prete st . 
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critsrici of phvsicQl QDPGQr*3ncs Qnci poiss clustor tOQOthor on Q cominon 
factor.* Subdividing the pretest data into two groups, the A students 
and the non^A students , we can see for the non**A set a similarity of 
factor content on 10 of the 19 scale items. Note how eyid6noe_y logical 
rea soninq , organization , and preparation; imagination/ interest ,, 
attitude . enthusiasm and vocal inflection; and physical .appearan^ 
cluster on separate factors across the B, C, D, and F grade levels. 
Excluding the F students allows the association of pojse_and 
physical appearance for the BCD grade group. The additional factor 
identifiod by the A students appears to be one involving the criteria 
of diction , fluency, and word choice . The other grade levels did 
not appear to be able to extract this factor consistently when trying to 
discriminate between speech qualities. 

In terms of accountable variance (the total amount of variance 
accounted for by the appropriate factor solution) , there was a perfect 
positive correlation between grade levels and percentege. However, 
the difference between the B, C, D, and F three factor solutions does 
not appear meaningful. An examination of a three factor solution 
for the A students revealed the same relationship but with only a 



*There is sufficient rationale for boing suspicious of the data stemming 

from the F students since (1) they represented a very small sample size, 
and (2) the explanation of failure in a public speaking course often has 
little to do with the demonstrated speaking proficiency of the student. . 
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slight difference between the A and B students . It would appear then 
that in terms of accountable variance the difference between grade 



levels is only a function of the number of factors exuacted from the 



data. 



4.19 Posttestl Analyses for Proficiency Levels. 

Posttestl analyses stem from data of student evaluahon of 
the filmed speeches during the final week of beginning courses in 
public speaking. These students did not participate in the pretest 
but viewed the same films shown at the time of the pretest at the 
particular institution . 

Tables 4.19-1 through 4.19-5 represent the posttestl factor 
analyses for A, B, C, D, and F students respectively. In terms of 
the number of identifiable factors for the five grade levels it can be 
seen that for both the A and B students a four factor solution appears 
most appropriate while for the C, and F students the three factor 
solution was most meaningful. 

In terms of the content of the factors there appears a similarity 
of factor structures between the A and B students . It should be noted 
from Tables 4.19-1 and 4.19-2 that 16 of the 19 scale items clustered 
on similar factors with respect to content. This can be seen by 
examining the factor loadings of evidence . logical reasoning . 
organization, and preparation; diction . fluency and word choice; 
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imagination . interest, facial expression , enthusiasm, attitude . total 
effect and vividness; physical appearance and eve contact . Tables 
4.19-3; 4.19-4; and 4.19-5 indicate a similarity of factor loadings 
across the three dimensions for the C; D; and F students of 11 of 19 
items; organization and preparation; attitude , enthusiasm . facial 
expression , fluency , imagination . interest and vividness; physical 
appearance and poise. Excluding the F students; a comparison 
between the C and D students reveals agreement on 17 of the 19 scale 
items with respect to the content of the established factor structure 
(the exceptions being the criteria of diction and word choice) . Across 
all five grade levels similarity in factor content seems limited to 
organization and preparation; imagination , interest , attitude , vividness 
and facial expression; and physical appearance. Excluding the F 
students the across grade level similarities are evidence , organization , 
logical reasoning and preparation; total effect, attitude , enthusiasm, 
facial expression , imagination, interest and vividness ; and physical 
appearance . It would appear that there was a high degree of similarity 
between the A and B students and the C and D students both in terms 
of the number of factors and the respective content of those factors 
and that the major difference between the AB subgroup and the CD 
subgroup was the former's ability to include diction, fluency and 
word choice o n a factor distinguishable from the other scale factor 



loadings . The data coming from the F students in this phase of 



the project do not seem to fit well into either the AB or CD 



subgroups . 

In terms of accountable variance there appears little 
difference between the totals for A and B students • Each of the 
identifiable factors also seems to account for like percentages of 
variance in their order of extraction. This would be expected 
given the high degree of similarity in the content of the factors 
for both the A and B students. With respect to the C, D, and F 
students it can be seen that the D's account for a higher total 
variance with their 3 factor solution than do the C and F students. 
However, this difference seems attributable to the fact that 
the first extracted factor for the D students represents considerably 
more scale items than the C and F students* On the second and 
third extracted factors, those for the C, D, and F students seem 
highly similar both in content and accountable variance. 

4.20 Posttest2 Analyses for Proficiency Levels 

Posttest2 analyses stem from data evolved from students who 
participated in the pretest. These students were shown a different 
set of films from the one they saw in the pretest during the final 
week of their beginning courses. In total the data in postteSt2 
evolved from student evaluations of all the films represented by 
posttestl. Since those students who did participate in posttest2 
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saw two sets of films while those who participated in posttestl saw 
only one set, the researcher decided to analyze the data from the 
two conditions separately.* 

Tables 4.20-1 through 4.20-5 represent the posttest2 factor 
i! lalyses for A, B, C, D, and F students respectively. Again, as 
in the case of posttestl data for the A and B ^ycudents a four factor 
solution seemed most appropriate while for the C, D, and F 
evaluators a three factor solution appeared most meaningful. 

In terms of the content of the factors an examination of Tables 
4.20-1 and 4.20-2 reveals a similarity between A and B students on 
17 of the 19 rating scale items. These include the following clusters 
of evaluative criteria on separate dimensions: evidence, logical. 
reasoning , organization and preparation; diction , fluency and word 
choice 2 imagination , interest , facial expression / onthusiasm, eye contact., 
attitude , vividness and vocal inflection; and physical appearance and poise_. 
Tables 4.20-3, 4.20-4, and 4.20-5 indJcate-a similarity of 
factor loadings across the three dimensions for the C, D, and F 
students of 11 of 19 items: logical reasoning , organization and 

*Because of various logistical problems not all the students who 
participated in the pretest were able to be involved in posttest2. A 
particular check was made, however, to insure that those who did 
participate in posttest2 had also been included in the pretest. Nine 
hundred forty eight students participated In posttest2, as opposed 
to 1811 who participated in the pretest. 
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preparation! imagination . interest, attitude, enthusiasm, facial 
expression, and vividness ; physical appearance and poise . 

Excluding the F students . a comparison between the C and D 
students reveals agreement on 13 of the 19 scale items.* Across 
all five grade levels there is similarity xn factor content on the 
following item clusters: logical reasoning , organization . preparation; 
imagination , interest . facial expression , enthusiasm, vividness . 
attitude; and physical appearance and poise . Excluding the F 
students allows the inclusion of the criterion evidence with logical 
reasoning . organization and preparation across all five grade levels. 
In terms of factor structure it appears that the students who 
participated in posttest2 can be dr' ded into two groups, the AB 
group and the non-AB groups. The distinguishing difference between 
the two groups seems to be the fact that the AB group was-able to 
identify a dimension represented by the criteria diction . fluency and 
word choice w hile the non-AB group was not, . This finding seems 
consistent with the results of posttesti. 



*In the case of posttest2 there is less rationale for the exclusion of 
the F students than in posttesti, since to have participated in this 
phase of the project would demand that the student complete a course 
in public speaking at least to the final week of classes. However, 
there remains the fact that the number of F grades received by the 
students who participated in posttest2 was so small as to yield 
suspicious data. 
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In terms of accountable variance it appears that the 
extraction of the fourth factor by the A and B students allowed 
them to account for more scale variance than the C, and F 
students , An examination of three factor solutions for the A and 
B students shows accountable variance of 74% and 73% respectively. 
This still represents a slight increase over the percentage of variance 
accounted for by the three factor solutions for the C, and F 
students (whose total variance varies only one percentage point) . 

4.21 Advanced Analyses*'for Proficiency Levels 

The advanced analyses stem from data evolved from students 
enrolled at the institutions in courses for which beginning public 
speaking was a prerequisite to enrollment. 

Tables 4.21-1 through 4.21-4 represent the advanced factor 
analyses for A, B, C, and D students respectively. No student who 
participated in the advanced phase of the project received an F grade. 
For all four grade levels a four factor solution seemed most appropriate. 
It will be noted, however, that for the B students in the advanced 
courses only one criterion, eve contact loaded extremely high on 
the fourth extracted factor. This makes the four factor solution in 
violation of one of the criteria advanced in Section III of the report 
(that a factor have at least two items with their highest loading on it) . 
For the purposes of interpretation this violation was allowed since 
it supported a high number of agreements across the four grade levels. 
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With respect to the content of the factors it can be seen 
that there is agreement between A, B, and C students in the advanced 
courses on 15 of the 19 scale items. This is represented by the 
loadings of evidence , logical reasoning . organization and preparation; 
imagination , interest, facial expression , enthusiasm . bodily movement, 
attitude . total effect, vividness a nd vocal inflection; physical appearance 
and poise. It is interesting to note that the factor represented by 
diction . fluency a nd word choice d oes stand for the A and C students 
as a separate factor but not as such for the B students. One possible 
explanation of this finding might lie in the wide range of qualities 
represented by the B students in advanced courses , Approximately 44% 
of the advanced students received a final grade of B, Adding the D 
students to the comparison reduces the agreement between grade 
levels to 10 of the 19 items , These include evidence, logical reasoning . 
organization and preparation; attitude . enthusiasm, facial expression 
and imagination ; physical appearance and poise. Again, one can be 
legitimately suspicious of the effect of the D students because of 
the small number who received this designation. 

In terms of accountable variance the four factor solution for the 
A students appears lo represent a slightly higher percentage than for 
the B, C, and D students. It is worth noting that in all conditions 
under v/hich data were collected it was the A students who accounted 
for the highest percentage of the scale variance. 
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4.22 Scale Item Reliability Across Proficiency Levels. 

As was the case with the determination of scale item 
reliability across training levels the statistic employed on the 
data separated according to proficiency levels was that of the 
Intraclass Correlation. The data used for analysis were, however, 
transformed in that the coefficients reported represent means of the 
reliabilities established by students in the various combinations of 
institutions, film-sets, film showings, orders and to some extent, 
training levels.* 

Table 4.22-1 represents the Intraclass Correlation 
coefficients for the 19 scale items for the beginning and advanced 
students at all proficiency levels. It will be noted that for the 
beginning students all grade levels are represented, but that no 
students in the advanced courses who participated in the project 
received an F grade . Because of the fact that not all grade levels 
were represented by the two types of students further analysis 
was limited to the effects of proficiency within the categories 
"beginning" and "advanced". Using the technique of analysis 
of variance it was found that there was a difference in the reliability 
coefficients given by the grade levels for the beginning and advanced 
students who participated in the project ( .0005). However, there 

*It will be noted from Table 4.22-1 that the reliability coefficients 
represent the grade levels of two classifications of students - beginning 
and advanced. This means that for the beginning students the mean 
reliability estimates reflect both the pretest and the posttestl conditions 
of the research . 
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was no significant interaction between proficiency level and scale 
item. It was also found that advanced students at the various 
proficiency levels tended to have lower reliability than their 
counterparts in the beginning courses. 

4.23 The Interaction of Training and Proficiency in Public 

Speaking on the Dimensionality of Speech Evaluation. 

In order to discuss the effects of the interaction of training 
and proficiency in public speaking on the dimensionality of speech 
evaluation it will be necessary to review the data as summarized 
in previous articles. This time an attempt will be made to examine 
each grade level through the four conditions under which data were 
gathered . 

Before going into the effects of training and proficiency 
in terms of the interaction of the two variables, the reader should 
remember that interaction is not being discussed in its true 
statistical sense. Such is really far beyond the scope of a factor 
analytic study. Instead interaction is here approached as an 
attempt to find a consistency of factor structure or structures across 
the logical combinations of essentially nominal classification 
systems. For the purposes of reporting the results of this quest 
for consistency, the following discussion will be subdivided according 
to grade levels and within each subdivision comparisons will be 
made among the various training levels (pretest, posttestl, posttest2. 
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and advanced) . 

4.24 The A Grade Evaluator Across Training Levels 

Table 4.24-1 represents the means and standard deviations 
for the A grade evaluators across the four conditions under which 
data were gathered. There appears to be no consistent pattern 
of differences between the pretest and posttest means and standard 
deviations for the A grade evaluators in the beginning courses. As 
noted previously, however, the students receiving A grades in the 
advanced courses consistently gave mean scores below those given 
by the students in the beginning courses, regardless of the training 
level represented by the prerequisite. With respect to standard 
deviations there appears to be no difference attributable to training 
in public speaking for the A grade evaluators. 

Tables 4.18-1, 4,19-1, 4.20-1, and 4.21-1 represent the 
factor analyses for the A grade evaluators across the four conditions 
under which data v/ere gathered. In each condition a four factor 
solution was deemed most appropriate. With respect to the content 
of those factors it can be seen that 16 of the nineteen evaluative 
criteria cluster in a similar fashion across the four conditions of 
the research. These groupings are evidence . logical reasoning , 
organization , preparation; total effect, attitude . enthusiasm, facial 
expression, imagination , interest . vividness a nd vocal inflection; 
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diction . fluency and word choice ; physical appearance# With respect 



to the three items not loading highest consistently across training 
levels (eye contact , bodily movement, and poise) there appears no 
reason to cite training levels as the explanation. In other words, 
when comparing the A students before a basic course; the A student 
after a basic course and the A students after an advanced course 
the same 16 items evolve a similar factor structure. 

With respect to accountable variance aga^n it can be seen 
that there is no consistent difference for the beginning students 
reoardless of when the data were gathered. The highest percentage 
of the variance accounted for by a four factor solution was for 
students in posttest2. It was these students who had two attempts 
to use the rating scale and this added experience with it might have 
accounted for the higher percentage of variance. The lowest 
percentage of variance was for the A grade evaluators in the advanced 
courses. This difference is so slight as to suggest that it is more 

attributable to the confounding effects of eve contact, bodily ^ 

< 

movement and poise than any training level. With respect to the 
variance accounted for by the particular factors there appears to be 
a consistent rank ordering across the training levels. The fluctuations 
in the variances of these individual factors again seem more attributable 
to the variables eve contact s bodily movement and pois^ thah to 



training. 



On the basis of the evidence cited there appears for the 
A grade evaluators no reason to suspect an effect on the dimensionality 
of speech evaluation due to the interaction of training and proficiency 
in public speaking. 

4.25 The B Grade Evaluator Across Training Levels 

Table 4,25-1 represents the means and standard deviations 
for the B grade evaluators on the 19 evaluative criteria across the 
four conditions under which data were gathered. As in the case of 
the A grade evaluators , there appears no consistent pattern of 
differences between the pretest and posttests means and standard 
deviations for the evaluators in the beginning courses. Again, the 
B grade evaluators in the advanced courses assigned consistently 
lower ratings to the films they viewed than did the students in the 
beginning courses, but the variance across all four conditions under 
which data were gathered does not seem to differ in any recognizable 
fashion. 

Tables 4.18-2, 4.19-2, 4.20-2, and 4.21-2 represent the 
factor analyses for the B grade evaluators across the four conditions 
under which data were gathered. It will be noted that the pretest 
ddua yielded a three factor solution. Both the posttests yielded 
four factor solutions. Between the two posttests there was agreement 
on 16 of the 19 scale items. The exceptions being total effect, 
eve contact and bodily movement. The inability of eve contact 
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and bodily movement to consistently cluster on an identifiable factor 
was also observed for the A grade evaluators. It appears from an 
examination of the factor structures represented by Tables 4.18-2, 
4.19-2, and 4. 20-2 that the B grade evaluators were able to clearly 
identify as a dimension of speech evaluation the criteria diction, 
fluency and word choice after having taken a basic course in public 
speaking. Table 4.21-2 reveals that the B grade evaluators enrolled 
in the advanced courses were not able to distinguish diction, fluency 
and word choice as a separate factor. With respect to the evidence- 
logical reasoning-etc. , the imagination-interest-etc . and the 
physical appearance-poise f actor there is general agreement among 
the B grade evaluators across the training levels.* 

In terms of accountable variance it can be seen that for the 
beginning speech students the identification of a fourth factor of 
speech evaluation at the conclusion of the basic course enabled them 
to account for a higher percentage of the scale variance. The fact 
that the B evaluators in the advanced course seemed to have difficulty 
in holding to the content of a fourth factor for evaluation is also 
reflected in terms of accountable variance . It will be noted that 
for the advanced students the four factor solution accounted for 

*Thesc labels are assigned on the basis of the strongest loadings 
of items on a particular factor and are not meant to imply any 
theoretical consideration. 
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about as much of the scale variance as the three factor solution 
for the beginning speech students prior to their taking the basic 



course in 






The rank order 



V-/X 



XOV^tUiO 
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of their contribution to the total accountable variance seems 
consistent across all training levels for the B grade evaluators. 

On the basis of the evidence cited above there does appear 
to be an effect on dimensionality due to the interaction of proficiency 
and training in public speaking. It appears that for the B grade 
evaluator a beginning course in public speaking contributes to the 
dimensionality of speech evaluation in the respect that he is able to 
make such evaluations on the basis of four factors rather than 
three. The fourth factor on which discrimination between film qualities 
seemed to be made was one in\felving the evaluative criteria of 
diction . fluency and word choice. In this respect one might conclude 
that the B grade evaluator moves in the direction of the A grade evaluator 
as a result of a basic course in public speaking. It is interesting to 
note that the B grade evaluator represents approximately 31% of the 
beginning speech population. The B grade evaluator in the advanced 
course seems to provide almost an uninterprecable factor structure. 

The safest conclusion appears to be that there is no consistent 
evidence to indicate an effect on dimensionality tha-i- can be 
attributed to taking an advanced course in public speaking for B 




grade evaluators 
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4.26 The C Grade Evaluator Across Training Levels 

Table 4.26-1 represents the means and standard 
deviations for the C grade evaluators for the 19 criteria 
represented on the rating scale. As in the case of the A 
and B evaluators there appears no consistent pattern of difference 
between the pretest and posttests means and standard deviations 
for the evaluators in the beginning courses. Again, the C grade 
evaluators in the advanced courses assigned consistently lower 
ratings to the films they viewed than did their counterparts in the 
beginning courses, but the variance across all four conditions 
under which data were gathered does not seem to differ in a 
recognizable fashion. 

Tables 4.18-3, 4.19-3, 4.20-3, and 4.21-3 represent the 
factor analysis for the C grade evaluators across the four conditions 
under which data were gathered. It will be noted that for all data 
involving the C grade evaluators in the beginning speech course a 
three factor solution seemed most appropriate. This particular 
solution shows consistent item clustering around three factors for 
15 of the 19 evaluative criteria contained on the rating scale used in 
the research. The same factor structure seems to hold for both . 
conditions of the posttests. The items which appear not to represent 
similarities of factor structure for the beginning students as the two 
stages of their training (before and after a course in basic public 
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speaking) a e total effect , diction . eye contact , and fluency.^. 

The reader will note that the criterion of eve contact was a consistent' 
problem for all proficiency levels with respect to the establishment 
of factor structure . Furthermore , it is only with the C grade 
evaluation that we have been able to find to date the clear 
establishment of a diction . fluency . word choice factor as' a result 
of having taken a basic course in public speaking. In comparing 
the factor structure evolved for data stemming from the C grade 
evaluator in the advanced courses we do find a. four factor 
solution to be meaningful. Furthermore, the C grade evaluators in the 
advanced courses appear to be able to identify the diction, fluency, 
and word choice factor. With respect to the evidence-logical reasoning- 
etc. factor and the imagination-interest-etc_. factor, the C grade 



evaluator is not really much different from his counterpart in the 
beginning course regardless of training level. It should be noted, 
however, that the noise -physical apDearance factor previously identified 



does not appear to hold for the C student in the advanced course. 
The criterion physical appearance does link with eye contact to form 
an identifiable factor. It does appear that there is a significant 
relationship between the three factor structures of the C grade 
evaluator in the advanced courses and the B grade evaluators in the 
posttes+1 condition of the research. Both of these student groups 
saw the same film s ts at their respective institutions and both 
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evolved factor structures of four dimensions with agreement on 48 

of the 19 scale items. This finding can be explained on the rationale 

that it is, the students who do relatively well in basic courses in 

public speaking who enroll in the advanced courses. The close 

parallel between the factor structures of the A and B students at the 

conclusion of a basic course in public speaking to that of the C. 

grade evaluator in the advanced course is evidence for this rationale. 

This finding also points to a lack of effect attributable to advanced 

1 

training in public speaking with respect to the dimensionality of 
speech evaluation. It is interesting that the criterion which seems 
to lack consistency in a comparison of the A and B evaluator at the 
end of the basic course in public speaking with the C evaluator 
in the advanced course is the item of poise. 

With respect to accountable variance there seems little 
difference between the solutions at different training levels for 
the C grade evaluator. For the three factor solution the individual 
factors have the same rank order across training levels. For the 
advanced C students the diction . fluency a nd word choice dimension 
replaces the poise -physical appearance d imension as the third 
strongest factor. In comparing the C grade evaluator to the A and 
B evaluator in the advanced courses we see less scale variance 
accounted for. This is also the case in comparing the C evaluator 
to the AB evaluators at the conclusion of the beginning courses . 
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- despite the fact that the content of the factors is very similar. 

jf 

The most warranted conclusion with, respect to the C grade^ 
evaluator is that there is no evidence to indicate a significant effect 
on the dimensionality of speech evaluation due to training in public 
speaking. The fact that the C grade evaluator in the advanced 
courses does pick up .a fourth factor is most reasonably explained 
by the idea that it is those students who do well in a basic course 
^who then enroll in advanced courses. 

/ 

i 

! 

4.27 The D Grade Evaluator Across Training Levels 

Before making an attempt to determine the effects of the 
interaction of proficiency and training in public speaking on the 
dimensionality of speech evaluation it should be pointed out that 

d 

the number of students receiving the D grade in the beginning 
courses totaled but 6.7% of the population and in the advanced 
courses 2.7%. In all instances the evolved factor structures 
represented less than 100 students at each training level. Such 
small sample sizes often yield less than consistent factor 
structures . 

Table 4.27-1 represents the means and standard deviations 
for the D grade evaluators on the 19 criteria across the four conditions 
under which data were collected . As in the other comparisons there 
appear to be no consistent differences between the pretest and 
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posttBsts for th6 students in the beginning courses. The tendency 
for the D student in the advanced courses to rate the filmed speeches 
below their counterparts in the beginning courses is still apparent 
but less consistent across the 19 evaluative criteria than with the 

previously described grade levels. 

Tables 4.18-4, 4.19-4, 4.20-4, and 4.21-4 represent the 
factor analyses for the D evaluators across the four conditions 
under which data were gathered. It will be noted that a thre© factor 
I olution seemed most appropriate for those D students in the 
beginning speech courses. Across the training conditions for the 
beginning speech students receiving D grades there appears to be 
agreement with respect to factor structure on. 11 of the 19 evaluative 
criteria contained on the rating scale. These include evidence^ 
logical reasoning, organization and preparation; attitude.^ enthusiasm ., 
facial expression , imagination and interest; physical appearance., 
and poise . In comparing the two sets of data for the D. students 
after they completed the basic course we see agreement on 12 of the 
19 factors. The increase over the pretest is the loading of the single 
it0ni vividness on the interest-imaginati on-etc_ dimension . This degree 
of consistency is below that observed at the other grade levels . 

In any event there seems to be no consistent evidence that the 
beginning course has any sufficient impact on the factor structure 
of D grade evaluators. For the D grade evaluator in the advanced 
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courses a four factor solution seems most appropriate. However, 
the structure is quite inconsistent with respect to other factor 
structures observed for students in the advanced courses, ^ftqain, 
this finding is invited given the small number of students who 
received D grades in advanced courses and given that the D 
evaluator represents the lowest proficiency level given. Thus, while 
the evolution of a fourth factor for the advanced D students is a con- 
sistent finding, the content of those factors with the exception of 
the evidence-logical reasonina-etc . one seems to defy comparison. 

With respect to accountable variance there appears to be an 
incident in which the beginning speech students receiving D grades 
who saw the film which had been used in the pretest as their posttest 
(posttestl) were able to account for a larger percentage of the scale 
variance than those students who participated in both the pretest and 
posttest (posttest2). However, the inconsistency of factor content 
makes this result somewhat suspicious. The same conclusion seems 
justified with respect to the ordering of the individual factors in 
their contribution to ihe total amount of accountable variance. 

For the D students there seems to be no consistent contribution 
of training at the beginning or advanced level to the dimensionality 



of speech evaluation. 



4.28 - The F Grade Evaluator Across Training Levels 

The F grade evaluator.in the beginning public speaking 
course represents^less than two percent of the total sample used 
for the research. There were no F grade evaluators in the advanced 
courses . Again this relatively small sample size really works 
against the meaningful interpretation of data in any comparative 
sense. The material that follows will be limited to mere description 
of results ob|:ained by running the statistical analysis required by 

I 

I 

the project on data representing the F proficiency level. 

Table 4.28-1 represents the means and standard deviations 
for the F grade evaluators in the beginning speech courses. One 
easily can see that the means and standard deviations for the F 
grade evaluator seem to vary more than for any previously described 
proficiency level. With respect to the means there appears no 
consistent pattern that would indicate an effect due to training. 

The standard deviations do show a change with respect to those 
students receiving F grades who participated in both the pretest 
and the posttest (posttest2) . It will be noted that there is a 
definite and consistent construction of the standard deviations 
in the posttest2 condition. However, this finding is inconsistent 
with the posttestl condition for students receiving an F grade at the 
same training level. An examination of the data reveals a possible 
explanation of this finding. The F students in the posttest2 
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condition, did not extend across all the institutions rtor all the film 

% 

sets used in the study. This latter fact could well explain the 

r 

findina with resnect to the standard deviations. 

' > 

Tables 4.18-5, 4.19-5, and 4.20-5 represent the factor 
structures for the F grade evaluators across the training levels for 
the students enrolled in the beginning courses. For all conditions 
under which data were gathered the F proficiency level evolved 
a three factor solution. Across the training levels for the beginning 
students there was agreement with respect to factor content on only 
6 of the 19 scale items . These were organization and preparation; 
imagination . interest, fluency a nd enthusiasm. In comparing the 
two posttest conditions for the F grade evaluator we see agreement 
on 12 of the 19 scale items. These items include logical reasoning, 
interest, imagination , facial expression , evidence, enthusiasm and 
attitude; preparation-, vocal inflection, organization . ey e contact and 
total effect. It is interesting that these comparisons involve only 
two of the three identifiable factors. With respect to factor structure 
there seems to be no consistent evidence of an effect due to training 
on the F grade evaluator's percept^n of speech evaluation. 

With respect to accountable variance there is a slight 
increase in the total percentage of those beginning students who 



complete the basic course and still receive an F grade. Because 



of the lack of consistency in regard to factor content it seems 
unwise to draw any conclusion with respect to"the rank order of 



the factors in terms of their contribution to the total variance 



Because of the small sample size, the fact that there 
were no F grade evaluators in the advanced courses, and the 
lack of factor consistency it would appear that there is no evidence 
to support an effect due to training on the dimensionality of speech 
evaluation for the F grade proficiency level. 



4.29 Scale Item Reliability for Proficiency Levels Across 

Training Levels. 

The data used in the determination of the scale item 
reliability in this section of tne report represent the mean 
Intraclass Correlation coefficients for:the various combinations 
of institutions , film-sets, film snowings , and orders while 
maintaining the divisions representing both training and proficiency 
levels . The number of actual coefficients used to determine tne 
mean reliability estimates varied, particularly with the sample 
sizes and film showings at the participating institutions . Howevei , 
each item of data represented the composite of at least six Intraclass 
Correlation coefficients . 

Table 4.29-1 represents the reliability estimates for the 19 
scale items for each proficiency level across the test conditions. 

It will be noted that both posttestl and posttest2 conditions represent 
data gathered from students who had completed a basic course in 
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I public speaking (training level 2) . In addition , it will be recalled 

% 

that there were no F students in the advanced condition of the 
research (training level 3). In order to further analyze the reliability 
estimates it v/as necessary to; (1) use the posttestl data as repre-- 
sentative of training level 2; and (2) examine only the proficiency 
levels that extended across all three training levels {A, B, C, and 
D students). Using a modified analysis of variance technique it 
was determined that there was a significant degree of interaction 
between training and proficiency levels in terms of scale item 
reliability ( .0005). This result would be expected given the 

results discussed in articles 4 .' i 6 and 4 . 22 . The safest conclusions 
that can be made are that in general those students most affected 
by a basic course in public speaking (the B and C proficiency 
levels) had slightly less reliability in using the project rating form 
after the course than before. For the students least affected (A and 
D) there was a general increase in reliability as a result of the 
basic course training. Furthermore, additional training (as repre- 

r 

sented by an advanced course) tended to result in a lowering of 
reliability for the B, C, and D students across the 19 scale items. 
These results fairly well parallel those of the effects of training 
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and proficiency on factor structure.* 

4.30 Summary of Section 

A 

The conclusions based on the results cited in this section 
will constitute the basis for the final section. It is enough to say 
here that care was taken in the processing of data to maintain those 
divisions which would best reflect the objectives of the project. 
While to some extent the maintenance of these divisions has 
resulted in a redundant narrative, the reader might be reminded that 
such repetition is an inherent byproduct of factor analytic research. 












*As a check on these findings the same statistical analysis was 
performed on the mean reliability estimates using the posttest2 
data as representative of training level 2. The results are similar 
to those discussed above except that for the B grade evaluator in 
the advanced condition there was no significant drop in reliability 
when compared to his counterpart at training level 2. 
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TABLE 4.16-1 

SCALE ITEM RELIABILITY ACROSS TRAINING LEVELS 
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TABLE 4.18-1 

PRETEST FACTOR ANALYSIS FOR A GRADE EVALUATORS 
4 Factor Solution - 591 Observations 
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CONCLUSIONS 
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5.1 Purpose of the Section. 

<? ^ 

The purpose of this portion of the report is to interpret the 

results cited in the previous section in light of the stated objectives 

of the research (see Section III! Article 3.5). ' 

5.2 An Interpretation of Results of the Research. 

The study involved approximately 4800 students (this number 
excludes the 800 plus students who participated in the project pilot 
study) enrolled in 27 different speech courses involving public 
speaking at eight institutions of higher education. The courses in 
which data were collected were divided into two classifications: basic 
or advanced. This classification rested- on the requirements that (1) 
the basic courses concentrated on public speaking and served as 
prerequisite to advanced courses also involving practice in public 
address, and (2) the advanced courses required a prerequisite basic 
course in public speaking and involved training in public address 
beyond that represented by their prerequisite. The job of applying 
these requirements to the courses in which data were collected fell 
to the institutional contacts who aided the project. 

Questionnaires were administered to all students who 
participated in the study in order to determine their previous training 
in public speaking. In general, students enrolled in those classified 



as basic courses were freshman or sophomores with little to no 
training in public speaking either at the high school or college level. . 
In contrast, students enrolled in the advanced courses tended to be 
juniors or seniors with at least a basic course in public speaking at 
the college level (usually taken at the same institution in which they 
were taking the advanced course) . 

Data were ccUected on students representing three training 
levels . Training level one data came from students prior to their 
completion of a basic public speaking course in college. Training level 
two data came from students who had just completed a basic course. 
Training level three represented data taken from students who had just 
completed an advanced course in public speaking. Within each training 
level data were subdivided according to the final grade each student 
received in his respective course. These grades (A, C, D, and F) 
were viewed as indicative of proficiency levels in public speaking 
as viewed by the instructors using the normal evaluative techniques of 
the particular courses. It is recognized that the evaluative techniques 
used in a public speaking course may not all be based on student 
performances as public speakers, but it was assumed that all the 
techniques used would have reference to the student's perception of 
what constitutes an effective use of oral communication,, whether 



by himself or others. 
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The prime objective of the research was to investigate the 

’ » ' i 

effect of training and proficiency in publife speaking on the dimensionality 
of speech evaluation. Dimensionality was defined operationaHy through 

Ji, 

factor structure, that is, the number of dimensions evolved via factor 

analysis for a particular set of data, the content of those dimensions - 

as measured by the strength of association betv/een relevant criteria 

♦ 

and the dimens ion^(f actor loadings) , and the strength of the dimensions 
in their ability to account for the total variance o- the criteria within a 
particular context involving student evaluation of public speaking. 

The gathering of data required the students at the three training, levels 
to evaluate three films of students giving public speeches in a 

• \ 

classroom situation. The speeches viewed in each set represented 

three nominal quality classifications termed low, middle, and high.* 

There were four different film sets used in the project. Each set 

was evaluated by students representing the various proficiency 

levels at each training level. Finally, each film set was viewed at 
< 

two or more of the participating institutions. 

Prior to the investigation of the main effects of training and 
proficiency on the dimensionality of speech evaluation, an attempt 
was made to establish that public speaking was viewed as multi- 
dimensional by the students who were about to do the evaluating of 

*;The film order was varied with each showing at each of the 
participating institutions . 



the hlmed speeches . In order to support such a conclusion the 
students prior to viewing the filmed speeches were asked to evaluate 
themselves aS public speakers. These self-perceptions were in the 
form of ratings on the 19 evaluative criteria that would eventually be 
used CO evaluate the filmed speeches . The results of the factor analyses 
of the self-perception ratings indicated a multi-dimensional view of 
public speaking on the part of the students regardless of training 
level. The results also support the conclusion that training in public 
speaking, particularly that represented by a basic course, does 
affect a student's self-perception as a public speaker. The srfdent 
tends to rate himself higher on the evaluative criteria used in the 
study. Furthermore, the student at the conclusion of a basic course 
in public speaking can evaluate himself in a fashion similar to that in 
which he evaluates others as public speakers. These results appear 
to extend beyond those evaluative criteria normally associated with 
self-confidence. There was no consistent indication of a significant 
effect of the self-perception data attributable to advanced training in 
public speaking. 

The results of the factor analyses of the self-perception ratings, 
then, do support the conclusion that students perceive the evaluation 
of public speaking to be multi-dimensional and that they are able to 
reflect this impression in their application of the evaluative criteria 
(rating scale items) developed for the project to themselves. 

•i \ 
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The researcher was also interested in establishing the relation- 
ship between dimensionality and the quality of speaking being 
evaluated. In order to determine this relationship separate 
factor analyses were made of the student evaluations of the low, 
middle, and high speeches. The results support the conclusion 
that the rating scale remained multi-dimensional in all the research 
conditions across the three nominal quality classifications of speaking 
represented in each film set. This result, coupled with that represented 
by the self-perception scales, points to the existence of dimensions 
of speech evaluation recognizable by students evaluating themselves 
or others . 

The most consistent finding of the research was the evolution 
of four factors of speech evaluation. To some extent this finding 
might well be a function of the particular set of scale items evolved 

♦ 

for the project. Another rating instrument with different or additional 
items might well evolve a greater number of factors applied by students 
with the same degree of consistency found in this study. It is 

obvious, however, that there is more to the evaluation of public 

\ 

speaking in the eyes of students than mere comments on "delivery" 
and "content". Students at all training levels seem ready for more 
sophisticated critiques of public speaking than they are normally 
given in the classroom. This finding should be welcom-^d by the 
speech teacher who has lamented the apparent superficiality of his 
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critiques in the eyes of students. ' 

In investigating speech students at^the three training levels 

in the research it was established that at the conclusion of a basic 

/ 

public speaking course four dimensions of speech evaluation were 
identified in the process of distinguishing between the qualities of 
public address represented in each film set. Three of the four evolved 
factors paralleled the dimensionality of evaluation used prior to the 

t 

training. What emerged at training level two could be labelled a 
"language facility" dimension of evaluation. This factor was represented 
by the scale items d iction . fluency and word choice . There was some 
evidence to suggest that students perceived such a factor prior to the 
basic course, but they were unable to consistently identify it as a 
dimension distinguishing between qualities of public address until 
after a basic college speech course . It is important to note that the 
emergence of the language facility factor did not destroy the dimen- 
sionality that the students brought to the speech course, but rather 
can be viewed as an additive effect of training in public speaking at the 
beginning level. The evidence of an additive effect represented : y 
training level three is not pronounced. While it is possible to evolve 
a five factor solution for the speech evaluations of students at the 
conclusion of an advanced course, it cannot be done without interfering 
with the factor content of the dimensions represented at training level 
two. The most plausible explanation of this result is that the advanced 
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courses at the various institutioi providing data for the project 
include a wider range of content than their basic courses and that 
this range confounded the factor structure when the attempf was being 
made to establish an additive effect of that level of speech training. 

This result points to the need for additional research into the differential 
effects of particular groupings of advanced courses with similar content 
on the dimensionality of speech evaluation. 

An investigation of the effects of proficiency in public speaking 

on the dimensionality of speech evaluation yields an interesting 

conclusion. It would appear that for the A students training in public 

speaking has little to no effect on how they evaluate the speaking of 

others . The students who appear to profit most in terms of a basic 

speech course are those who receive B grades. As a result of the 

basic course the B students begin to view speech evaluation in a manner 

similar to the A students. The C, D^, and F students reflect no significant 

degree of change from pretest to posttest on the dimensionality of 
*/ 

speech evaluation. In general, it is the ability of the AB students 
to use the language facility dimension as a fourth factor for speech 
evaluation that distinguishes them from the CDF students at the end 
of a basic speech course. The C student in the advanced courses 
does support the establishment of the language facility factor in a 
manner similar to the A and B students at the end of a basic course 
in public speaking. It is hypothesized that this result does not 



suppprt an effect due to training in the advanced course, but can be 
explained by the hypothesi'3 that the enrollments in ad7anced courses 
are comprised to a large extent of students who did well in the basic 
course. It should be remembered that the students in the advanced 
courses tend to rate themselves as B public speakers. 

The results of the study do not tend to support a conclusion 
that training and proficiency offer a type of differential effect on the 
dimensionality of speech evaluation. 

To many the results of this study will be disappointing. Certainly 
very little can be claimed for the effects of training and proficiency in 
public speaking in terms of the dimensionality of speech evaluation. 
However, even this finding can provide some saving grace for the field 
if it will stimulate a more concerted effort on the part of its educators 
to use their theory and training to develop in students a firmer and 
broader basis for the evaluation of public speaking than they appear 



to have at the present time. 
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APPENDIX A 



COURSE DESCRIPTIONS OF PARTICIPATING UNIVERSITIES - HEW PROJECT 

MICHIGAN STATE yNIVERSITY 

Speech 101 - Public Speaking. Principles and practice in effective 
speaking in both formal and informal situations. 

Speech 116 - Group Discussion . Principles and practice in effective 
lerdership and participation in group discussion and 
conference. 

Speech 305 - Persuasion. Study of and experience in the process of 
influencing human behavior through persuasive oral 
communication. 

Speech 309 - Argumentation . Types of argumentation employed in our 
society, including forms of debate. Field trips to the 
state legislature and courts in Lansing and vicinity. 

Speech 401 - Speech for the Classroom Teacher. Basic requirements 
for the teacher's oral communication, and consideration 
of speech activities utilized in classroom instruction. 

IOWA STATE UNIVERSITY 

Speech 30 - Basic Elements of Public Speaking . Basic elements of 
public speaking, including content and organization; 
practice in the preparation and delivery of speeches, 

f 

Speech 311 - Fundamentals of Speech. Fundamental principles of public 
speaking, audience analysis, interest and attention, 
selection and organization of speech material, delivery. 
Practice in preparation and delivery of extemporaneous 
speeches . 

Speech 312 - Business and Professional Speaking . Methods of application 
of fundamental principles of public speaking to composition 
and delivery of common types of business and professional 
speaking; practice in preparation and delivery of various 
types of speeches. 

Speech 334 ~ Persuasion. Principles and methods of persuasive 
speaking; discovery and use of evidence; proof; 
refutation; appeals; organization. Practice in 
preparation and delivery of persuasive speeches 
upon topics of current interest. 
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Speech 336 - Group Discussion . Practice and procedures of 
problem-solving groups; communication theories 
related to group procedure; group leadership and 
participation. 

UNIVERSITY OF WISCONSIN 

Speech 101 - Fundamentals of Speech. Consideration of the process 
of oral communication and the fundamentals of speech, 
with application of these fundamentals in selected 
speaking, discussion, and oral interpretation projects. 

Speech 103 - Public Speaking. Study and application of the principles 
of public speaking in informative, persuasive, and 
special occasional speech situations. 

Speech 105 - Public Sneaking. Development of fundamentals in the 
preparation, delivery, and evaluation of the common 
forms of public address. 

Speech 262 - Argumentation and Debate . The theory of argument with 
practice in the preparation and delivery of various types 
of argumentaxive speeches and debates. 

Speech 464 - Theory and Practice of Persuasion. Consideration of 
principles, processes, and methods of persuasion 
with practice in the preparation and delivery of various 
types of persuasive speeches; includes critical and 
creative problems in both oral and written forms . 

OHIO STATE UNIVERSITY 

Speech 401 - Effective Sneaking . The principles of effective speaking. 

Preparation and presentation of informative and persuasive 
speeches. The speech processes with emphasis on speech 
as a thinking process. 

UNIVERSITY OF CONNECTICUT _ 

Speech 111 - Fundamentals of Speech. Theory and practice in the 

principles of communication; identification of purpose, 
organization, supporting materials , audience analysis, 
style and delivery. 

Speech 112 - Public Sneaking. A thorough study and practice of the most 
commonly used types of speech with emphasis on audience 
analysis and persuasion. 



o 






-A3- 



UNIVERSITY OF MINNESOTA 

Speech 5 - Fundamentals of Speech. Development of basic skills 

in meeting a variety of speech situations: extemporcmeous 
speaking, oral reading, discussion. Development of 
basic understanding of speech processes and forms. 

Speech 51 - Advanced Public Speaking. Preparation and delivery of 
speeches on current public issues. 

UNIVERSITY OF ILLINOIS 

Speech 101 - Principles of Effective Speaking. Preparation and 

presentation of short informative and persuasive speeches 
with emphasis on the selection and organization of 
material, methods of securing interest and attention, 
and the elements of delivery. 

Speech 121 - Advanced Public Speaking . The Logical Bases of 

Discourse. Study of theory of argument, e.g. , evidence, 
reasoning, and construction of briefs; practice in 
formal and informal forms of debate and public discourse 
on current public questions. 

\ 

m 

UNIVERSITY OF MICHIGAN 

Speech 210 ~~ Types of Public Speaking . Consideration of special 
problems in exposition, argument, and persuasion. 
Preparation and delivery of speeches representative 
of solutions to problems in invention, style, and 
arrangement. 

Speech 211 - Parliamentary Procedure and Group Leadership .!. 

Consideration of the rules of procedure for both large 
and small groups , and the problems of presiding at 
meetings. 

Speech 410 - Principles of Discussion and Conference. A study 
of the fundamentals of effective leadership and 
member participation in small group meetings. 

Training experiences in varied types of discussions 
and conference.. . 

Speech 412 - Elements of Persuasion . A study of audiences, motivation, 
principles of attention and suggestion, use of emotional 
proofs, and bases of belief and action applicable to 
tc persuasive speaking. 
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Speech 413 - Advanced Public Speaking . Principles of individual 
and group behavior relevant to efforts at influencing 
such behavior through speech. Composition and 
delivery of speeches aimed at accommodation of audience 
attitudes toward the speaker's proposition. 
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INSTRUCTOR - SECTION COUNTS: PARTICIPATING SCHOOLS 



MSU 

Instructors = 30 
Sections (Winter Term) ' 53 
Sections (Spring Term) = 41 
Sections (Fall Term) = 45 

IOWA STATE 
Instructors == 15 
Sections =32 

WISCONSIN 
Instructors = 5 
Sections = 14 

OHIO STATE 
Instructors = 6 
Sections = 8 

CONNECTICUT 
Instructors = 6 
Sections = 13 

MINNESOTA 
Instructors = 20 
Sections = 28 

ILLINOIS 
Instructors =12 
Sections = 14 

MICHIGAN 
Instructors = 6 
Sections = 7 
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Outlines of HEW PROJECT SPEECH FILMS 



Speech Title or Subject: Advertising 
Speaker No: 4 
Quality Rating: Low 
Speech Outline: 



Introduction 

1 . How many of you own a telephone ? 

2 . How many of you own a T .V . ? 

3. How many would be in the market for a new portable T .V . 
for say under $50? 

4 . Tell of my experience with Huntington Music Adv . 

5 . Show Hanley Dawson Adv . 

6. What is the retailer trying to do when he advertises? 

Things to do in answering an ad. 

I. Keep a mental note of places you like to do business 
with. 

II. Check with those who know if you don't. 

III. Use that telephone. 

1. Verify salesman's name. 

2. Verify product specifications. 

3. Verify the price. 

IV. You can beat the retailer ^ (ie^ buy the traffic builder.) 
Research Used: 

My questionnaire showed that the class had much more faith 
in the retailer than I had expected. Up 'til now I had shown no 
actual bogus ads and feel that this is probably the most 
opportune time to do such and awake a few people . The 
appreciation for advertising is there according to question 
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responses, but some have a lot to learn about what 
advertising does. I don't want them to learn the 
hard way because this spoils a person even for 
the legitimate advertiser. 



Speech Title or Subject: The Plank Road Farm 
Speaker No: 6 
Quality Rating: Low 
Speech Outline: 

I. More Leisure 

A. Advantage 

B. Disadvantage 

C. Leisure as a problem 

II. Michigan Tourist Attraction 

A. Interesting but not exciting 

B. Not adequate 

III. Plank Road Farm 

A. What it offers that is different 

IV. Response Desired 

A. Recognition of problem of leisure . 

B. Tourist attractions are not adequate 

C. Plank Road Farm is more worthwhile 

V. Means of measuring response 

A. Show of hands when questioned 

VI. Use of questionnaire 

A. Determined degree of activeness 

B. Revealed preference to authentic attractions 

C. Failed to prove prejudice towards farmers, although I feel it 
does exist. 
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Speech Title or Subject: The Time Study Man 
Speaker No: 8 
Quality Rating: Low 
Speech Outline: 

1. Introduction - Explanation . 

2. Integrate into questionnaire statistics 

3 . Government control statistics on media 

4. Question each of the control items 

5 . Psychologists 

6 . Control over carmakers 

7. Kathy Miller's questions 

8. Give my answer 

9 . Try to elicit response 

. Speech Title or Subject: Corruption in Government 
Speaker No: 13 
Quality Rating: Low 
Speech Outline: 

I. Ethical system governing legislators 

A. There are none 

B. Attempts at systems have been made 

C. There are good Senators tod 

II. Audience opinion 

A. Less than one half think the problem is serious or very 
serious 

B. Over 60% of the audience think I think the problem is serious 

I 

C. More examples 

1. Cook County Illinois 




2. Michigan's own Senator Hart 
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III. Availability of information 

A. Not confidential 

B. Reader's guide 

n/. Desired effects of appeal 

A. No earth-shattering changes 

B. Encouragement 

1. 48% of us have an above average interest in government. 

2. Well over half of us think we can do something about tKe 
situation. 

C. Long-range attitude influence 



Speech Title or Subject: Ungraded School Systems 
Speaker No: 3 
Quality Rating: Middle 
Speech Outline: 

Introduction 

I. Some of you (42%) are going into teaching and 
administration. Those of you who aren't should 
still be concerned with the schools. 

A. You will be paying for the schools and teachers 
in your community. 

B. Sending your children to these schools. 

C. May be in a position to influence decisions as 
to school policy. 

Because 42% are going into education these people will have to face 
this problem in the future. The rest of the audience will be affected 
by this problem because it will touch them in the above ways . 

II. Passing judgment on educational practices had become 
the "Great American Pastime." 
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A. Some of this criticism is unjustified but some 
is valid. 

B. Schools like any other institution are not perfect. 

Because part of my audience is against ungraded schools I think 
it is better to concede that the present method isn't completexy 
wrong . 

III. Elementary and High Schools are for everyone. 

A. Average child loses'nothing by being classified. 

B. Poor students, bright and gifted and students with 
special abilities are often overlooked . 

C. Some say this is the consequence we pay for trying 
to educate the whole population . 

D. Other methods have been tried and are working 
the ungraded classroom. 

Body 

I. Children develop at different levels emotionally, physically, 
intellectually, and culturally. 

A. This creates complex problems for the teacher who 
must teach them all. 

II. An example of culturally deprived children being ability - 
grouped is the Head Start Program. 

III. Ability grouping has been criticized because it takes 
away factor of competition. 

A. Impossible for some students to compete with the rest 
of the class. 

B. Competition still involved but they compete at their 
own level. 

IV. Ability grouping has been criticized because it places a 
stigma on the poor student . 

A. Just as much of a stigma placed on them in a diversified 
classroom. 
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B. At their own level they may be more relaxed and 
willing to try harder. 

V. If the ungraded classroom works at the elementary level it 

will work in the high schools . 

A. Some of you well prepared for college but unfortunately 
most of us have not. 

B. This isn't only the results of methods— poor teachers, 
curriculum, out-moded equipment and poor judgment on 
our parts add to poor preparation, 

C. But teaching methods have something to do with it. 

D. Ungraded system offers opportunity to work, at own 

level — not retained or pushed ahead there is no 

threat of failure. 

1 . No fear of attempting courses because competition 
is too stiff, material covered too fast and is too 
deep. 

E. Few of us had opportunity to study a subject in depth. 

An ungraded system would have given us the opportunity. 
There would be no single textbook but the material 
would be geared to the student's capacity. 

F . Students would have more responsibility which is too 
often left for college years . 

VI. An example of a working ungraded high school is Cape 

Kennedy High School. 

A . Found to work with students anticipating college and 
those who only to graduate from high school. 



Speech Title: Aged Drivers 
Speaker No: 12 
Quality Rating: Middle 
Speech Outline: 

I. Summary of statements. 

A. Three causes of automobile accidents. 
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1. Car 

2. Road 

3. People 

B. 49,000 killed in 1965 in United States. 

C. Special concern - persons too old to drive. 

1. In 1965, 27,000 people killed over 65. 

2. Causes 

a. Poor vision 

b. Bad hearing 

c. Slow reflexes 

II. Awareness of problem is high. 

A. None disagreed-that safety items aren't worthwhile. 

B. 83% agreed there was problem of older driver. 

III. Solution law requiring all drivers over 60 years of age to 
take reflex, hearing and vision tests upon renewal of 
license. 

A. Would affect only those who couldn't pass test. 

B. Effective instruments for testing. 

C. Your part - support legislation. 

IV . Benefits 

A. Reduces chance of auto accidents. 

B. Safer for you and I. 

V. Closing: I was involved in an automobile accident with an 
elderly man. A person too old to drive. I never want it to 
happen to me or you, again. So when the time comes to 
do something about it - I think we should . 
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Speech Title or Subject: Physical Fitness 
Speaker No: 14 
Quality Rating: Middle 
Speech Outline: 

I. Introduction ” 

A. 37% believe that in order to be physically fit, must exercise 
frequently . 

1. Through exercise, could build a tremendous physique. 

2, In my point of view there would be something missing — 
enjoyment. 

B. In 2nd question, 89% of you agreed that exercise could be 
beneficial to one's mental, physical and spiritual health. 

1, Plain exercise doesn't help mental and spiritual but 
creative activity such as swimming does . 



II. Body 

A. 72% of you in class already have an interest in some activity. 

1, When one has an interest in something, it is most beneficial 

I 

to participate in it quite often. 

B. You might be wondering, what are the values of physical 
activity? (will look at it in the recreation sense) 

1. Health - develops muscles, stimulates circulatory system, etc. 

2. Can help prevent mental illness (release people from their 
problems) 

a. Develops positive attitudes through accomplishments. • 

3. Educational values - learn skills, develop an appreciation 
for beauty which in turn helps build character. 

a. Not just an escape from education but a vitalizing 
factor in the process itself. 

b. Outdoor recreation brings us in close contact with 
nature (ex. Plank Road Farm and Eve) 



5 . Citizenship needed for group activities . 



6. Individuality - means of releasing creative power, 

a. ex. of veterans . 



Speech Title or Subject: North Central Accreditation. 

Speaker No: 16 
Quality Rating: Middle 
Speech Outline: 

Introduction: The last time I spoke to you, I talked about how the 
N.C.A. was organized and the goal of the organization - to promote 
higher standards of education. To accomplish this goal, the N.C.A. 
set up an evaluative criteria divided into three main areas: teacher 
certification , curriculum planning and facility planning . Tonight 
we are going to look at the N.C.A. , its goal and its methods to see 
if it accomplishes its purpose. 

I. Teacher Certification 

A. Subject background 

B. Experience 

\ 

1. Albert Einstein 

2. Noithwood Institute 

a. Faculty 

b. Graduate successes 

c. Credits transferable 

d. Experience 

C. Professional status of teachers 

1. Implications 

2 . Credit for economy 
a . Requirements 



b. Degrades students 



II . Curriculum Planning 



A . Sta nd ardi z ation 

1. Results 

2. Questioning 

a. Survey 

b. Needs 

3. - Experimentation 

a. Nancy World 

b. Experimental schools 

c. Individual study 

4. Effect on democracy 

a. Educations importance 

b. New ideas 

III. Facility planning 

1. C.C.H.S. - Personal Experience 

a. unaccredited 

b. waste of resources 

c. blessing in disguise 

d. student success 

1. statistics 

2. conclusion 

IV . Methods 

1. Outside advice 



a. Education's purpose 
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b. N.C.A. dictator 

2. New needs -flexibility 

a. Business oriented 

b. Qualified evaluators 

3. Local School Systems and N.C.A. Policy 

Conclusion: I know some of you like myself are already marrried and 
chances are pretty good that the rest of you will be. I have a four 
month old daughter who just got her first tooth trday . It was a big 
event and it made me realize she is growing up. As a parent y. I want 
the best for her like you do or will for your children. We've been 
sold a bill of goods on this N.C.A. and now is the time tc start doing 
something about it. We must take the initiative to insure that our 
children get the best education possible. Don't allow education to 
be held back by some authoritarian organization. Encourage your 
school board to use other means besides N.C.A. dictates to evaluate 
our schools. We want the best. The choice is up to us. 



Speech Title >or Subject: The New Morality 
Speaker No: 1 
Quality Rating: High 
Speech Outline: 

I. The only thing new about the "New Morality" is the title 1 The 
basic code has been present in our society ever since time 
began y along with two other forms of moral decision-making. 

The other two codes have both had their time of prominence - 
they have been discussed y questioned y applied y and eventually 
receeded back to a place of lesser importance. Today the "New 
Morality" is a major concern. I have already given you statistics 
and information showing that the trend today is in favor of the 
New Morality. Now I'd like to explain a little of the theoretical 
background and show you how you can take an active part in this 
revolution . 

II. The three basic moral decision-making policies are as follows: 

A. Legalism 

1 . Rules are absolute and unyielding . 
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2. Obedience to prefabricated "rules of conduct" is 
more important than freedom to make responsible 
decisions . 



3 . Common in older generation 



a. 


Close family ties 


b. 


Strict discipline 


c. 


Strict religion 


Antinomian or law-less method 



1. No rules and no guidelines - spontaneous decisions 

2. Post war period - lost generation 
C. "New Morality" “ situation ethics 

1. A form of ethical relativism 

a. People have moved to this because moral laws 

tend to be ambiguous both in theory and application. 

b. People enter into each and every decision-making 
situation armed with principles like a legalist 
but are prepared to suspend or violate any rule 

if the situation so warrants. 

2. Ethics based on love rather than law. 

III. Each of these philosophies has had a period of dominance, and 
really it is impossible to draw a line showing where one slows 
down and the other takes over. 

A. All three philosophies are present today, (example - divorce 
issue in class) 

B. There is no one reason for why one code becomes more 
popular at a given time . One explanation for why one 
stops and the other starts could be as Justin Lewis , 
sociologist, says: 

"When people become used to a certain way of 
thinking or when a new, startling approach becomes 
a normal commonly accepted belief, people will 
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follow it for a while but will some become 
bored by it. They will question other 
attitudes and often, at this point, the 
previous common attitude will be replaced 
by the new, challenging concept. " 

IV. By examining our society today, as I have partially explained 
previously, one can see many changes that have taken place 
and that indirectly helped pav.e the way for the New Morality. 

A. Mobility - key characteristic today of American society. 

1. Community ties are almost nil.. 

2. Church influence often weak. 

B. Partially because of the mobility factor, family ties are 
very weak in a majority of cases . 

1. On the survey taken in this class alone, 58% of 
class answered that their family was not very close. 

2 . Parents of the post w?^r period have tended to relax 
discipline and often replace it with complete freedom. 

a. Low value systems - vandalism 

b. Confusion 

c. Parents feel they are being kind to their kids by 
giving them a free reign in all matters, but noted 
child psychologists have stated that kids need 
and respect discipline. 

C' 

(relate to survey) 

C. Our society is sex-oriented. All ages are pre-occupied with 
sex and usually are unashamed to dis<"uss it. 

1. Sex education - responsibility o' parents. 

2. Parents often fail (survey) 

V, So now you know the issues. You have had the main theory of 
the New Morality explained , you have heard a few ideas on why 
things occur as they do, and you have seen how you yourselves 
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are taking part in this revolution. 

I know that my speeches haven't forced all of you to weld 
into a mass that will want to go out and fight against the 
New Morality. 

But I don't want you to do this. 

Instead, I ask only one thing of you. Regardless of what 
your position on the New Morality is, DISCUSS IT WITH 
AT LEAST ONE OTHER PERSON (behavioral response ) . Make 
that person aware of the issue, and make him take an interest 
in it. 

By doing this , you are playing an important part in the morality 
revQlHtion . By lifting the mystery that surrounds this code , 
the code itself will become dull and common, and it will 
recede to its proper place of lesser importance as have the 
codes of anomism and legalism. 



Speech Title or Subject: The Peace Corps 
Speaker No: 2 
Quality Rating: High 
Speech Outline: 

A. Two speeches available 

1 . For a grade 

2 . Because I wanted to 

B. Persuasion 305 

1. Original objectives 

a. Persuade you to join 

b. Affect attitudes positively 

2. I hope I fail in the first objective. 

C. Too important a decision for a series of speeches. 

D. Decision took me a long time. 

1. Writings about the Peace Corps 
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2. Speaker in class two years ago. 

3. Sister joined two years ago. 

4. Letters home. 

a. Lost on a trip 

b. School protests 

c. Delivering a baby 

d. Hopes and frustrations 

5. Peace Corps booth. 

6. Test and acceptance for training. 

7 . Training . 

8. Still not decided. 

E. I can't persuade you to join. It's too important a decision. 

F. There is no 'Peace Corps'. 

1. There is a collection of volunteers . 

2. There is a collection of individuals. 

3. There is a collection of individual successes and 
failures . 

G. I chose the wrong topic because I'm too involved to be 
objective. 

H. Rather than my original objectives I hope this to be just 
a part of a gradual persuasion process . 

I« I also hope to personalize the Peace Corps. 

J. Jobs that need doing — visual aids. 

K. You can't make the Peace Corps yourcareer but part of your 
career could be the Peace. Corps . 
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Speech Title or Subject: The Art of Cookery 
Speaker No: 15 
Quality Rating: High 
Speech Outline: 

K 

I. Many scientists predict that in the future, perhaps the next 
twenty years , we may be living on pills along to fulfill our 
nutritional requirements . 

B . Monotony . 

C. Food experience in class. 

1. 42% small variety . 

2. 16% fair variety. 

3. 37% good variety. 

4. 05% large variety. 

5. 00% exceptional variety . 

D. Might swallow that pill now-monotony . 

II. Need for adequate knowledge increases. 

A. Bachelors. 

1. Live alone. 

2. Don't want to eat out every night. 

B. Women 

1. Dire necessity. 

2. Will not learn principles from "Better Homes & Gardens." 

3. Learn scientific principles, easy and enjoyable. 

C. Entertaining. 

1 . Social - He who invites his friends and then neglects 
to give his personal attention to the food served to 
them, does not deserve to have them. 
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a. Plan banquet. 

b. Dinner at home. 

c. Hostess or host-must carve meat. 

2. Business purposes . 

D. Nutritional need for variety and quality. 

E. Satisfaction received. 

1. Feel proud. 

2. Artist conception. 

III. Educational experience*. 

A. Each meal is a new experience. 

B. Not a bitter pill to swallow. 

C. Does not need to be that way for you. 

D. It does not matter where you learn the scientific principles 
of cookery. It is just vital that you do. Education in 

the composition, preparation, and service, of fine food is 
necessary for you. My only advice: Get out and learn! 



Speech Title or Subject; Auto Safety 
Speaker No: 18 
Quality Rating: High 
Speech Outline: 

I. 50,000 lives have been taken by automobile accidents this year 
and over one million men and women have been disabled. 

A. The Federal Government has taken steps to reduce traffic 
accidents but their power is limited. 

B. Twenty-one states have safety inspection laws. 

1. Those states which do not have safety inspection 
laws have 6.07 fatal accidents per 100 million miles 
of driving. 
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2. Those states which have safety inspection laws 
which are not administered by the State Highway 
Department have 4.74. 

3. Those states which have safety inspection laws 
which are administered by the State Highway 
Department have 2.83 fatal accidents per 100 
million miles of driving. 

4. The National fatality rate is 5.20 fatalities per 
100 million miles. 

II. The Pennsylvania state inspection system is administered by 
the State Highway Patrol. y 

A . The State Highway Patrol registers qualified service 
stations , garages , and new car dealers . 

B. The State Highway Patrol trains and registers mechanics 
who will be permitted to make motor vehicle inspections. 

C. Every licensed vehicle in the State of Pennsylvania must 
be. inspected every six months. 

D. Only items which can cause traffic accidents are inspected. 

1. External lights and headlights must be working and aimed 
properly . 

2. Horn must sound. 

3 . Windshield wipers must be functioning and blades must 
be good. 

4. Brakes must function within standards of safety. 

a. The pedal must travel freely. 

b. Brake wheel cylinders, master cylinders, brakes, 
hoses and lines must be free of leaks and must 
meet requirements of wear. 

5 . Exhaust system is checked for leaks . 

6. The tires are checked for wear and abrasions . 
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7. The sheet metal is inspected for protrusions. 

8. All glass must be free of .'•acks which can be 
felt by the hand . 



III. The signing of petition will help reduce fatalities and injuries 
on Michigan's Highways. 

A, The Department of State can use petitions such as these 
in their presentation to the State Legislature this fall. 

B, The adoption of this program will not cause hardships 
on operators of motor vehicles because vehicles should 
be inspected every 6,000 miles. 

C, $5.00 per year is minimal if Michigan's fatalities can 
be reduced from 6.07 to 2.83 per 100 million miles 

of driving . 




Appendix C 



Department of Speech 

Michigan State University 

U.S. Office of Education Project #6-1767 

OEC-3-7-061761 

East Lansing, Michigan 

Dear Sirs: 

In consideration of my participation in the research project 
entitled The Effects of Training and Proficiency in Public Speaking 
on the Dimensionality of Speech Evaluation I hereby authorize you 
to record on film, tape or otherwise, my visual and audio likeness 
and performance, and to use and to authorize to use such recordings 
or films within the context and definition of the cited research 
project. 



Signed: 



Witnessed: 



Date: 
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Rate each performance 
on a seven point scale 
with i low and 7 high. 
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November 2 , 1966 



Herbert J. Oyer, Chairman 
Department of Speech 
Michigan State University 
East Lansing, Michigan 

Dear Dr. Oyer: 

The intent of this letter is to determine the willingness of your 
department to participate in a research project sponsored by the 
United States Office of Education. This project, dealing with 
the dimensions of classroom speech evaluation, is being administered 
through Michigan State University. The scope of the research is 
to involve several universities and colleges engaged in undergraduate 
speech education. It is this purpose which prompts this letter. 

The objectives of the research are three in number; 

1) To determine specific dimensions of speech evaluation which can 
be used to make pbjective distinctions between varying qualities 
of public speaking. 

2) To determine the nature of the relationship between training in 
public speaking and the recognition and use of dimensions of 
speech evaluation which make objective distinctions between 
varying qualities of public speaking. 

3) To determine the nature of the relationship between developed 
proficiency in public speaking and the ability to recognize and 
use dimensions of speech evaluation which make objective dis- 
tinctions between varying qualities of public speaking. 

The basic strategy of the project is to have students enrolled in public 
speaking courses evaluate sets of filmed speeches using a multi-dimen- 
sional rating scale. The film-sets and the rating scale have been 
prepared at Michigan State University. Each set of three films repre- 
sents "good”, "average", and "poor" examples of public speaking as deter- 
mined by the Rhetoric and Public Address staff of the MSU Department 
of Speech. Basic Public Speaking enrollees will evaluate two sets of 
films (early in the course-late in the course) . Students in advanced 
courses (requiring a prerequisite basic public speaking course) will 
be asked to evaluate two sets of films at the conclusion of those 
QQyj*g0g ^ y^here possible we would like to have the students involved 
in this project use the developed rating scale in the evaluation of 
actual classroom assignments. 



In addition to the rating scale, a questionnaire has been developed 
to determine the amount/type of speech training that a given student 
has had prior to participation in the project. Within the question- 
naire is a section whereby the student can make a self-evaluation of 
his proficiency as a public speaker. The student's grade in the 
courses involved in the project will also be used as a measure of 
speech proficiency. 

The rating scale and questionnaire are on IBM scoring forms so as to 
be machine processed. We assume that the acquisition of course grades 
can be handled through the institution's record keeping division. 

All data will be processed at Michigan State University. The major 
statistical techniques to be employed will be factor analysis and 
factorial-discriminant, analysis. Data from each participating insti- 
tution will be processed separately and results made available to 
that participant. Results of the total project will also be made 
available . 

In terms of the actual administration of the project at participating 
institutions we would like to have it handled by a competent member 
of the staff. To this end we are willing to pay for such help. In 
combination with, or as an alternative to this suggested method of 
administration, we are willing to send to participating institutions 
members of the project staff to aid in the gathering of data. Again, 
all costs associated with the project will be assumed by Michigan 
State University. 

We hope that this letter serves to stimulate your interest in the project 
and that your department would be willing to cooperate with Michigan 
State in accomplishing its objectives. We realize that a more detailed 
schedule of events is necessary before you can make a definite commit- 
ment. What we would most appreciate is for you to designate a member 
of your staff whom we could contact on a personal basis at the upcoming 
SAA convention. At that time we should be able to determine how the 
logistics of this project can be incorporated within the academic calen- 
dar of your department. 

Sincerely, 



Herbert J. Oyer, Chairman 
Department of Speech 



William B. Lashbrook 
Project Director 



Letters dated November 2 , 1966 and signed by Dr. W. B, Lashbrook: 



Herbert J. Oyer, Chairman 
Department of Speech 
Michigan State University 
East Lansing, Michigan 



Jack Matthews, Chairman 
Department of Speech 
University of Pittsburgh 
Pittsburgh 13 , Pennsylvania 



Karl R. Wallace, Chairman 
Department of Speech 
University of Illinois 
Champaign Urbana, Illinois 



J. Jeffery Auer, Chairman 
Department of Speech 
Indiana University 
Bloomington, Indiana 47401 



H. Clay Harshbarger, Chairman 
Department of Speech 
University of Iowa 
Iowa City, Iowa 



James H. McBumey, Dean 
School of Speech 
Northwestern University 
Evanston, Illinois 



W. Hayes Yeager, Chairman 
Department of Speech 
The Ohio State University 
Columbus, Ohio 43210 



William M. Sattler, Chairman 
Department of Speech 
University of Michigan 
Ann Arbor, Michigan 

Ray E. Nadeau, Chairman 
Department of Speech 
Purdue University 
Lafayette, Indiana 



Frederick W. Haberman 
Department of Speech 
University of Wisconsin 
Madison, Wisconsin 



Kenneth L. Graham, Chairman 
Department of Speech 
University of Minnesota 
Minneapolis, Minnesota 55455 
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Connecticut 



Illinois 



Iowa State 



Michigan State 



Minnesota 



Ohio State 



Pittsburgh 



Wisconsin 



INSTITUTIONAL CONTACTS 

Dr, William Arnold 
Department of Speech 
University of Connecticut 
Storrs, Connecticut 06268 

Dr. Robert L. Ince 
Assistant Professor 
Department of Speech & Theatre 
University of Illinois 
136 Lincoln Hall 
Urbana, Illinois 61801 

Mr. Edward Bodaken 
Department of Speech 
Iowa State University 
Ames, Iowa 50010 

Dr. William Lashbrook 
Department of Speech 
Michigan State University 
East Lansing, Michigan 

Dr. David H. Smith 
College of Liberal Arts 
Department of Speech, Comm. & 
Theatre Arts 

University of Minnesota 
Minneapolis, Minnesota 

Dr. Eldon Baker 
Department of Speech 
The Ohio State University 
154 North Oval Drive 
Columbus, Ohio 43210 

Dr. Otis Walter 
Department of Speech 
University of Pittsburgh 
Pittsburgh, Pennsylvania 

Dr. Lloyd Bitzer 
Department of Speech 
University of Wisconsin 
Madison, Wisconsin 53706 



AC 203 

429-3311 

EXT-1200 



AC 217 
333-3617 



AC 515 
294-4111 



AC 517 
355-6690 



AC 612 
373-2851 



AC 614 
293-6558 



AC 412 
621-3500 



AC 614 
262-2543 
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Dr. Howard Martin, Chairman 
Department of Speech 
University of Michigan 
Ann Arbor, Michigan 48104 



AC 313 
764-5350 
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Appendix F 

Procedures for Data Collection 
Speech Rating Project, Phase I 
(Michigan State University) 



Films 



Three 800 feet films are provided. Each film shows three 
student speeches which are to be rated on the rating form 
provided. Each film shows the same three speeches but 
in a different order . The films are labelled Film I, Film II, 
and Film III. 

Method of Showing Films 

Of your classes in beginning Public Speaking which are 
involved in this project, one half will see a film at the 
beginning and at the end of the course, and the remaining 
half will see a film only at the end of the course. 

Those classes which see a film at the Beginning o f the 
course (Phase I) will see either Film I, or Film III. One 
third of them should see Film I, one third should see 
Film II, and one third should see Film III. 

For example, if twelve of your classes (sections) are 
involved, at the beginning of the course you would have 
a schedule such as the following; 

2 classes see Film I at beginning of course 
2 classes see Film II at beginning of course 
2 classes see Film III at beginning of course 
6 classes see no film at beginning of course 

The films should be shown by 16mm sound projector with 
a 2 inch lens . 

The projector should be stopped at the end of each speech 
to give subjects a chance to fill out the rating form for 
the respective speech. 

Rating Forms 

The rating forms ask for biographical data and also provide 
spaces for rating the speeches. 



The rating forms are numbered I, II, or III. Those numbered 
I should be used with Film I, those numbered II with Film II, 
etc. 

Subjects should be given time to fill out the biographical 
section of the questionnaire prior to the showing of the 

^4 1 w. 
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The rating form has spaces for rating four speeches but 
only three speeches will be shown. Disregard the space 
for the fourth speech. 

Abbreviations used in the biographical data section are: 



Question 9 



4. Scoring Pencils 



R & TV 
R & PA 
Sph. Ed. 
Sph. Sci. 
Thr. 



Radio & Television 
Rhetoric and Public Address 
Speech Education 
Speech Science (correction) 
Theatre 



Special scoring pencils are being provided. These should 
be used by all subjects in filling out the questionnaire 
and rating forms . 

5. Billina Procedures 



If expenses are incurred in connection with this project, a 
statement of the expenses should be sent to:' 

Michigan State University 
Account No. 71-2125 



6. Mailing Procedures 

The films and completed questionnaires should be returned 
by mail to the Department of Speech, Michigan State 
University, East Lansing, Michigan 48823. 



7. Call Us 



If you have any questions whatever about the project, please 
call us collect at Area Code 517, 355-6690, East Lansing, 
Michigan . (Brad Las hbrook or Murray Hewgill) . 
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Procedures for Data Collection 
■Speech Rating Project, Phase II 
(Michigan State University) 



Films 



Two sets of films, set and set , are provided. 

Film set includes 3 films labelled , , _ 

Film set includes 3 films labelled , , 



Each film shows 3 student speakers. Each set of films 
shows the same speakers but in different orders. 

Films in set are to be shown to subjects who did not 

see a film at the beginning of their course. 

Films in set are to be shown to subjects who did 

see a film at the beginning of their course. 

It is important that the appropriate rating forms be used with 
each film. 



Rating Forms 

The rating forms are labelled in 
They are divided into two sets, 

Set of the rating forms is 

and . 

Set of the rating forms is 

and 



a manner similiar to the films, 
set and set . 

subdivided into , , 

subdivided into . , 



Rating forms labelled should be used with the film 

labelled , etc. 



You will note that the rating forms in set include some 

biographical questions, while those in set do not. This 

is so because the biographical information has already been 
obtained from the subjects who saw a film at the beginning 
of the course. 



The rating forms have spaces for rating more than three 
speeches but only three speeches will be seen in each film. 
Disregard the spaces for rating the additional speeches. 



Abbreviations used in the biographical data section of rating 
form are; 

Question 9 R & TV 

R & PA 
Sph. Ed 
Sph. Sc 
Thr 

Method of Showing Films 
Beginning Courses 

The classes which did not see a film at the beginning of the 

course should see a film in set . One third of them 

should see film / one third film , and one third 

film . 

The classes which did see a film at the beginning of the 

course should see a film in set . One third of these ' 

should see film , one third film , and one third 

film . 

For example, if twelve of your classes (sections) are involved, 
you would have a schedule such as the following; 

2 classes see film . 

2 classes see film . 

2 classes see film . 

2 classes see film . 

2 classes see film . 

2 classes see film . 



Radio. & Television 
Rhetoric and Public Address 
Speech Education 
Speech Science (correction) 
Theatre 



Advanced Courses 



None of your advanced classes saw a film at the beginning 
of the course. One third of these classes should see film 

, one third film , and one third film . This , 

as you may guess, is to control for order effect. 

The films should be shown, as before, with a 16 mm sound 
projector, with a 2 inch lens. 

The projector should be stopped at the end of each speech to 
give subjects a chance to fill out the rating form for the 
respective speech. 
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4. Student Grades 

We will be in touch with you by telephone about the most 
appropriate means of obtaining from your department the 
final course grades of the students involved in this 
project. 

5. Scoring Pencils 

Special scoring pencils are being provided. These should 
be used by all subjects in filling out the questionnaire and 
rating forms . 

6. Billing Procedures 

If expenses are incurred in connection with this project, a 
statement of the expenses should be sent to: 

Michigan State University 
Account Mo. 71-2125 

7. Mailing Procedures 

The films and completed questionnaire and rating forms 
should be returned by mail to the Department of Speech 
Michigan State University, East Lansing, Michigan 48823. 

8. Call Us 



'Jt. 
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If you have any questions whatever about the project, please 
call us collect at Area Cdde 517, 355-6690, East Lansing, 
Michigan. (Brad Lashbrook or Murray Hewgill). 
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MICHIGAN STATE UNIVERSITY 

SPEECH COMMUNICATION RESEARCH LABORATORY 

September 1, 1967 

PROGRAM SCORE 

DATA TRANSFORMATION FOR MSU SPEECH RATING SCALES 
PROGRAM LANGUAGE: 3600 FORTRAN 

B. LASHBROOK 

This program has been tested by its contributor, however, no warranty 
expressed or implied, is made by the contributor or the Speech Com- 
^^^^ication Research Laboratory as to the accuracy and processing 
of the program and its related materials. Any questions concerning - 
this program should be addressed to Dr. Brad Lashbrook, Speech 
Communication Research Laboratory, Department of Speech, Michigan 
State University. 



This program was developed as part of a research project 
student evaluation of classroom speaking sponsored 
United States Office of Education. 



dealing with 
by the 
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PROGRAM DESCRIPTION 

SCORE is a procedure for transforming data, punched onto cards via 
an IBM 1230 optical scanner, into standard characters (one character 
per card column) and repunching converted data onto cards in a format 
determined by the user. This program was developed particularly 
for the transformation of data stemming from the MSU Speech Rating 
Scale. 

The program allows for 12 identification codes . Each card or set of 
original data can be read into the computer under the control of a 
user created format (INPUT!) . Data can be outputted in the form of 
punched cards under the control of a format (OUTPUTjJ created by 
the user. . . 

When this program is used to transform data stemming from an optical 
scan of a MSU Speech Rating Scale the speech scales will be ordered 
according to quality with the third identification variable determining 
the order of speeches (according to quality) at the evaluation time. 

The program contains three subroutines (FRONT, BACK, BACKl) . Each 
subroutine can handle a particular type of data coining off a scan of 
the MSU Speech Rating Scale. All data should be grouped according 
to the user's desire to use a particular subroutine. More than one 
subroutine may be used on any given run of the program. Data within 
these groupings may be in any order. 

An illegal character in any non -identification field will terminate the 
program. However, since the program processes one card at a time 
all material preceding the illegal character will be available . 

The program can process up to 100 variables. There is no limitation 
on the number of observations . 




If more than one 
subroutine is to be 
•used the bracketed 
group (cards 6-10) 
should be repeated. 

The NON -CHOICE card 
will terminate the pro- 
gram so it should be 
the last card of the 
particular run. 
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CARD PREPARATION 

Begin in 
Card # Column 

1 , 



2 1 



3 1 

4 7 



sS 



6 



1 



7 



1 



Contents 

PNC cards are prepunched and provided 
by the Computer Laboratory. 

I JOB, PROBLEM NUMBER, 6-character 
title, job time in minutes. Surname 
initial . 

I FTN, X 

y 

PROGRAM SCORE is a prepunched pro- 
gram and must be obtained from the 
Speech Communication Research 
Laboratory . 




CHOICE CARD 

The first two columns of this card 
designate the choice of subroutine to 
be used for the transforming of data. 

The following values correspond to 
the options now available: 

01 SUBROUTINE FRONT (43 Variables) 

02 SUBROUTINE BACK (69 Variables) 

03 SUBROUTINE BACKl (72 Variables) 

Any other symbol in the first two columns 
of the CHOICE CARD will terminate 
the program. 

INPUTi: A format description beginning 
with the first parentheses, of the data 
at time of input. This format is unique 
only to the data that follows it . All 
data should be processed as INTEGER 
values. 







8 1 OUTPUTi: A format description ^ 

beginning with the first parentheses, 
of the way in which the user desires 
to have the punched output arranged. 
This format is unique only to the 
data that follows it. All data should 
be outputted as INTEGER values. 

9 DATA in the format specified in on 

card 6 (INPUTi) 



10 1 DATA TERMINATION CARD: The 

symbol 9 in column 1 will terminate 
the processing-of-'data-by a particular 
subroutine and return control to the 
main program. The next card to be 
read should be a CHOICE CARD. 



11 1 NON-CHOICE CARD. Should contain 

in columns 1-2 some other symbol 
than 01, 02, 03. This will then 
terminate the main program. 

SAMPLE JOB ASSEMBLY 
PNC 

7 JOB, 999999, NOGOOD,!, Smith, B. 

I FTN,X 

PROGRAM SCORE 
I RUN, 1,100 

01 

(4311) 

(16 , 2X, 16 , 2X, 12 , 2X , 1011 , 2X, 1911) 

6210162323319787654317777 654831235555441111 
9 

00 
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PROGRAM SCORE 



PROGRAM SCORE 

TYPE INTEGER CHOICE, DATA 

DIMENSION DATA (100) 

COMMON DATA 
CALL OLDIOH 

1001 READ 1002, CHOICE 

1002 FORMAT (12) » 

IF (CHOICE. LT.l) GO TO 1003 
IF (CHOICE . NE . 1) GO TO 1004 
CALL FRONT 

GO TO 1001 

1004 IF(CHOICE.NE.2) GO-TO 1005 
CALL BACK 

GO TO 1001 

1005 IF (CHOICE. NE. 3) -GO TO 1006 
CALL BACKl 

GO TO 1001 

1006 PRINT 1007 

1007 FORMAT (lHO,*YOUR OPTION IS NOT AVAILABLE VIA THIS PROGRAM. 
1 PLEASE CHECK THE DESCRIPTION*) 

1003 END 

SUBROUTINE FRONT 
TYPE INTEGER DATA 
TYPE INTEGER INPUT, OUTPUT 
COMMON DATA(IOO) 

DIMENSION INPUT(10),OUTPUT(10) 

10001 READ 10002, (INPUT(I), 1=1, 10) 

10002 FORMAT (10A8) 

READ 10002, (OUTPUT(I), 1=1, 10) 

1 READ INPUT , (DATA (I) , 1=1 , 43) 

IF(DATA(1).EQ.9) GO TO 111 

DATA(-2-2)=4 

GO TO 11 

10IF(DATA(22).NE.4) GO TO 12 
DATA(22)=3 
GO TO 11 

12 IF (DATA (2 2). NE. 5) GO TO 13 
DATA(22)=2 
GO:TO 11 

13IF(DATA(22).NE.7) GO TO 14 
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DATA(22)=1 
GO TO 11 

14IF(DATA(22).NE.9) GO TO 15 
DATA(22)=0 
GO TO 11 
15DATA(22)=5 
li CONTINUE 
DO 3 1=25,43 
IF (DATA (I). LT. 2) 4,5 

4 DATA(I)=4 
GO TO 3 

5 IF (DATA (I) . NE . 2) GO TO 6 
DATA(I)=1 

GO TO 3 

6IF(L^TA(I).NE.3) GO TO 7 
DATA(I)=2 
GO TO 3 

7 IF (DATA(I) . NE . 4) GO TO 3 
DATA (I) =3 
3 CONTINUE 

PUNCH OUTPUT, ,((DATA(I),I=1,10),DATA(12),DATA(14),DATA(11),DATA(13 
1) ,(DATA(K),K=15,43)) 

GO TO 1 
lllRETURN 
END 

SUBROUTINE BACK 
TYPE INTEGER DATA 
TYPE INTEGER INPUTB, OUTPUTB 
COMMON DATA(IOO) 

DIMENSION INPUTB(10),OUTPUTB(10) 

10001 READ 10002, (INPUTB(I), 1=1, 10) 

10002 FORMAT (10A8) 

READ 10002, (OUTPUTB(I),I=1,10) 

1 READ INPUTB, (DATA (I), 1=1, 69) 

IF(DATA(1).EQ.9) GO TO 3 
DO 4 1=13 , 69 
IF (DATA (I). LT. 4) 5,6 

5 DATA(I)=2 
GO TO 4 

6 IF(DATA(I).NE.4) GO TO 7 
DATA(I)=1 

GO TO 4 

7 IF (DATA (I). NE. 5) GO TO 8 
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DATA(I)=:3 
GO TO 4 

8 IF(DATA(I).NE.6) GO TO 9 
DATA(I)=4 

GO TO 4 

9 IF(DATA(I),NE.7) GO TO 10 
DATA(I)=:5 

GO TO 4 

10 IF (DATA (I). NE. 8) GO TO 11 
DATA(I)=6 
GO TO 4 

11 IF (DATA (I). NE. 9) GO TO 4 
DATA(I)=7 

4 CONTINUE 

IF(DATA(3).EQ.l) GO TO 12 
IF(DATA(3).EQ.2) GO TO 13 
IF(DATA(3).EQ.3) GO TO 14 

12 PUNCH OUTPUTS, ((DATA(I) ,1=1,12) , (DATA(J) , J=14, 50 , 2) , (DATA(K) ,K=13 , 
1 49,2),(DATA(L),L=51,69)) 

GO TO 1 

13 PUNCH OUTPUTS, ((DATA(I) ,1=1,12) , (DATA(J) , J=13, 49 , 2) , (DATA(K) ,K=51, 
1 69),(DATA(L),L=14,50,2)) 

GO TO 1 

14 PUNCH OUTPUTS, ((DATA(I) ,1=1,12) , (DATA(J) , J=51, 69) , (DATA(K) ,K=14 , 50 
1 ,2),(DATA(L),L=13,49,2)) 

GO TO 1 
3 RETURN 
END 

SUBROUTINE BACKl 
TYPE INTEGER DATA 
TYPE INTEGER INPUTB1,0UTPUTB1 
COMMON DATA(IOO) 

DIMENSION INPUTBl(lO) ,OUTPUTB1(10) 

10001 READ 10002, (INPUTBl (I) ,1=1,10) 

10002 FORMAT (10A8) 

READ 10002, (OUTPUTBl(I), 1=1, 10) 

1 READ INPUTBl , (DATA (I) , 1=1 , 7 2) 

IF (DATA(l) . EQ . 9) GO TO 3 
DO 4 1=16,72 
IF (DATA (I). LT. 4) 5,6 

5 DATA(I)=2 
GO -TO 4 

6 IF (DATA (I). NE. 4) GO TO 7 
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DATA(I)=1 
GO TO 4 

7 IF (DATA (I). NE. 5) GO TO 8 
DATA(I)=3 

GO TO 4 

8 IF(DATA(I).NE.6) GO TO 9 
DATA(I)=4 

GO TO 4 

9 IF(DATA(I).NE.7) GO TO 10 
DATA(I)=5 

GO TO 4 

10 IF(DATA(I).NE.8) GO TO 11 
DATA(I)=6 

GO TO 4 

11 IF(DATA(I).NE.9) GO TO 4 
DATA(I)=7 

4 CONTINUE 

IF(DATA(3).EQ.l) GO TO 12 
IF(DATA(3).EQ.2) GO TO 13 
IF (DATA(3) . EQ . 3) GO TO 14 

12 PUNCH OUTPUTB1,((DATA(I),I=1,10),DATA(12),DATA(14),DATA(11), DATA(1 
1 3) , DATA(15) , (DATA(J) , J=17 , 53 , 2) , (DATA(K) , K=16 , 52 , 2) , (DATA(L) , L=54 

272)) 

GO TO 1 , 

13 PUNCH OUTPUTBl, ((DATA(I) ,1=1,10) ,DATA(12) , DATA (14) ,DATA(11) ,DATA(1 

1 3 ( , DATA(15) , (DATA(J) , J=16 ,52,2), DATA(K) ,K=54 , 7 2) , (DATA(L) , L=17 , 53 

2 , 2 )) 

GO '.TO 1 
3 RETURN 
END 



Appendix I 



! 



FTN5,3A 

PROGRAM TRANS P , 

TYPE INTEGER i 

TYPE INTEGER STOP J 

dimension data (iOO) ' 

READ 15, A J 

15 FORMAT (Alj 3 

2 READ l,STOP,(DATA(I),I=l/57) 

1 FORMAT (A1,15X,3(19F1,0,2X)) 

IF(STOP,EQ,A) GO TO 4 . ^ 

WRITE OUTPUT TAPE 44,3, (DATA(I) , 1=1 , 57) 

3 FORMAT (32X,19F1,0) 

GO TO 2 ^ 

4 ENDFILE 44 

REWIND 44 j 

END i d 

I ' 

1 

RUN, 10, 100 I 

1 o 

EXECUTION STARTED AT 1857 -28 i 

LOADMAIN,37,10,36000 



EXECUTION STARTED AT 1858 -33 
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Rate yourself as a public speaker on a seven point scale with 1 low and 7 high. 
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Your total effect : "" 
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